Need some help from MemGuard's debug gurus

Can anyone normally explain in humans words, what mena that error:

Quote:

It is point on start or end of allocation ? Can anyone give more info about what it can be and why ?

Thanks in advance.

@kas1e

32 bytes were allocated, but something tried writing more data to it, or didn't start at the beginning, hence overflowing the 32 byte allocation.

String buffers are typical causes of this with unterminated strings.

Simon

@Rigo

why this kind of error not detected on DSI itself (for having a better problem output for sending to the author) ???


I say that because sometime there are only memguard and sometime memguard first and immediately DSI..


I say that also because often programmer said: sorry, I don't found something and when a DSI came, the programmer found more easily the problem.

@Mrodfr

Quote:

"DSI" means "data storage interrupt". It just means that an access to an invalid address happened. This can be caused by a gone-wild application which writes to random addresses or by an application which writes just one byte beyond its allocated buffer limits and the buffer is right at the end of a memory page.

MemGuard can report such errors upon a Free#?() operation only, while the DSI may happen at any time as soon as the invalid access happens. But as the name "interrupt" implies it is an interrupt. Interrupts must be handled fast and you cannot perform fancy stuff like wall checks, symbol look up, etc. The GrimReaper performs this task outside the interrupt.

Application might even write beyond a buffer and you won't get a DSI at all, because the write access still happened within a valid memory page. Such accesses can only be detected by MemGuard. But MemGuard slows down system performance a lot, because the system does lots of memory allocations. Thus the system itself is doing only some very quick checks. Everything else must be examined when there is a demand for a check, but not all the time.

@Rigo
Quote:
So, it's overflow on writing of data to the buffer (i got that error, when i do glDeleteTexture opengl call by the way). But of course that minigl function itself can still write something on delete_texture stage ..

@tboeckel


Quote:
Did i understand right, that this error which i point, happenes on Free memory syscall ? (but memguard say on Allocation). I mean can you please a bit explain more, how exactly memguard got that error happenes ?



Quote:

Imho, will be cool to have just an options for GR, called "advanced debugging", where you can choice all the possible bugs catching features to works. So default users will not have that, but anyone who want develop something, will have it in background and will sure that all going well

Because for example, i am first time to know, that GR catch not all the bugs, and MemGuard can catch other ones, which, not so "light" and can cause a hard lockup in end.

@kas1e

Quote:

Of course MemGuard can capture such bugs on freeing only. How could it do that during the allocation? Think about it. A MemGuard hit during one of the Alloc#?() functions would mean the memory system itself is buggy and cannot even perform an allocation correctly. Especially something like damaged walls can only be detected if the application releases the memory back to the system.

MemGuard just remembers the parameters of the allocation call (function name, size, flags) and prints them during the free operation to give the developer a hint which allocation might have been affected. This information can be useful if the hits occurs with very certain allocations only.

@tboeckel

I'm not at all a programmer, but I betatest often.

I have been annoyed during lots of months about a beta AOS4 program, who just make memguard hit and nothing else.

this memguard hit fall, on the stack part,on nothing interesting (that have not helped the programmer) and thats all

I have accused Ibrowse, HTTPS, pthreads.library,.....
and finally just a normal beta program problem....

the programmer fall on a page (I remember by zero hero - compiling advice - don't remember the link) about an advice to compile with -gstabs.

I have received the next beta and test and surprising to see interesting DSIs !!!

sending the DSI to the programmer and he have added some check and no problem at all.


Here I think: memguard give wrong informations and DSI allways silent How can we make program efficiently

as kas1e, we need an advanced DSI settings with better tools, even make gdb working or add addr2line directly (or memguard).


As I have said, DSI not working allways and memguard seemed to give wrong informations (and -gstabs) seemed to have helped us at the end.


- C programmers and betatesters need a document for helping to program and debug well an AOS4 program ???

VALGRIND:

I have googled with-gstabs and have found a link who said for removing compiling error with valgrind, compiling with -gstabs (a googlecode link).

@Mrodfr

The memory allocation process uses 4096-byte pages. Those pages may be broken up into smaller "slabs", but once a 4096-byte page has been allocated from available memory, any reference within that 4096 bytes is legal and won't cause a DSI. You'll only get a DSI if you access outside the allocated page or pages.

Memguard intercepts your 32-byte request and adds a buffer zone ("wall") at each end (I don't know how much but probably just a single longword). It writes a known pattern into the space at each end of the 32-byte allocation. When that space is released at the end of the program, it examines the buffer zone at each end to see if the program wrote past the beginning ("front wall") or the end ("rear wall") of the allocation. Since the "walls" are probably within the 4096-byte allocated page, no DSI will occur.

@thread

thanks for the explanation about DSI and memguard

I think we need also a practice answer, for the programmers, mainly when receiving a memguard only hit, from a betatester.

- What they must do, to found and correct the error, based only with a memguard report ???

NOTE: I'm often diseapointed to have answer from programmer to said they can't found the problem with a memguard and maybe because there are missing things to know for using well a memguard report...


for example, this one (the thread about the USB error on this forum)

SFS: Warning: Task 0x5fe5a070 ("OHCI Controller Task Unit 0") called DoPkt()
SFS: Warning: Task 0x5fe5a070 ("OHCI Controller Task Unit 0") called DoPkt()
SFS: Warning: Task 0x5fe5a070 ("OHCI Controller Task Unit 0") called DoPkt()
SFS: Warning: Task 0x5fe5a070 ("OHCI Controller Task Unit 0") called DoPkt()
SFS: Warning: Task 0x5fe5a070 ("OHCI Controller Task Unit 0") called DoPkt()

MEMGUARD HIT - "OHCI Controller Task Unit 0" (5fe5a070)
allocation of 8192 bytes by AllocVecTagList() failed
0: 6f611be4 5fe3ad90 00000000 01ca97c4 00002000 5fe3ae18 59cb2c58 59563048
8: 00002000 00000008 01ca97c4 0000207f 24000082 00000000 00000000 00000000
16: 00000000 00000001 59563048 5a5ecb50 5fe23680 00002000 01e70000 5ff9fb40
24: 00000000 00000040 59c749a8 5fe954c8 00000000 00000040 00000002 00000003
----> 6f611be4 : "MemGuard" segment 0005 offset 8bdc
----> 01ca97c4 : "Kickstart/kernel" segment 0001 offset 497c0
----> 01ca97c4 : "Kickstart/kernel" segment 0001 offset 497c0
----> 01e70000 : "Kickstart/newlib.library.kmod" segment 0001 offset 061c
LR 014136f0 : "Kickstart/kernel" segment 0000 offset 136ec
CTR 6f611be4 : "MemGuard" segment 0005 offset 8bdc

Stack backtrace:
----> 014136f0 : "Kickstart/kernel" segment 0000 offset 136ec
----> 018c7abc : "Kickstart/ohci.usbhcd" segment 0000 offset 5558
----> 018c5274 : "Kickstart/ohci.usbhcd" segment 0000 offset 2d10
----> 018cc2c8 : "Kickstart/ohci.usbhcd" segment 0000 offset 9d64
----> 018cc448 : "Kickstart/ohci.usbhcd" segment 0000 offset 9ee4
----> 018c9d74 : "Kickstart/ohci.usbhcd" segment 0000 offset 7810
----> 0141587c : "Kickstart/kernel" segment 0000 offset 15878

Disassembly:
014136ec: 4e800421 bctrl
014136f0: 80010084 lwz r0,132(r1)
014136f4: 38210080 addi r1,r1,128
014136f8: 7c0803a6 mtlr r0
014136fc: 4e800020 blr

MEMGUARD HIT - "OHCI Controller Task Unit 0" (5fe5a070)
allocation of 16384 bytes by AllocVecTagList() failed
0: 6f611be4 5fe3ad90 00000000 01ca97c4 00004000 5fe3ae18 58f6dea8 59030040
8: 00004000 00000008 01ca97c4 0000407f 24000088 00000000 00000000 00000000
16: 00000000 00000000 59030040 5a1550c0 5fe23700 00004000 01e70000 5ff9fb40
24: 00000000 00000040 59c749d8 5fe954c8 00000000 00000040 00000081 00000003
----> 6f611be4 : "MemGuard" segment 0005 offset 8bdc
----> 01ca97c4 : "Kickstart/kernel" segment 0001 offset 497c0
----> 01ca97c4 : "Kickstart/kernel" segment 0001 offset 497c0
----> 01e70000 : "Kickstart/newlib.library.kmod" segment 0001 offset 061c
LR 014136f0 : "Kickstart/kernel" segment 0000 offset 136ec
CTR 6f611be4 : "MemGuard" segment 0005 offset 8bdc

Stack backtrace:
----> 014136f0 : "Kickstart/kernel" segment 0000 offset 136ec
----> 018c7abc : "Kickstart/ohci.usbhcd" segment 0000 offset 5558
----> 018c5274 : "Kickstart/ohci.usbhcd" segment 0000 offset 2d10
----> 018cc2c8 : "Kickstart/ohci.usbhcd" segment 0000 offset 9d64
----> 018cc448 : "Kickstart/ohci.usbhcd" segment 0000 offset 9ee4
----> 018c9d74 : "Kickstart/ohci.usbhcd" segment 0000 offset 7810
----> 0141587c : "Kickstart/kernel" segment 0000 offset 15878

Disassembly:
014136ec: 4e800421 bctrl
014136f0: 80010084 lwz r0,132(r1)
014136f4: 38210080 addi r1,r1,128
014136f8: 7c0803a6 mtlr r0
014136fc: 4e800020 blr

@Mrodfr

What you have to remember with things like the USB system, and shared libraries too, is that the code contained inside them is used by applications, or data is sent to a running process. If that data is somehow wrong, the cause could be a crash inside a library because it expects the data to be of a specific type, which it isn't. This will show up as a DSI inside, for example, intuition.library, but the real fault is in the code that called that function.

Running memguard is a good way to help in debugging, but the real power comes from understanding the stack trace and where in your code a particular function has been called.

If there really is a bug in any system library code, then there is every chance it will show up from multiple places. If only one application causes this crash, it's most likely the application feeding bad data. Remember the old saying: "Garbage in, garbage out" :)

There is very little that can be done to stop this during runtime, unless a system is implemented that marks a memory block as to be only used for a specific type of data, and this block contains "xyz". While this sounds like a good idea, the overhead would bring the system to a crawl.

Yes, we do need some better diagnostic tools for programmers to use, but it all boils down to someone having the patience, time and will to sit down and do it.

Simon