I’ve been looking for this for a while !! Why this f***ing Python process eat so much memory ? I freed all the objects, and don’t have any circular references ..
Every advanced Python programmer need to read this carefully.
On the other side, i’m looking for a good shared memory module for Py ? Do you have some links for me ?
Thanks for that link… I hope they post it on daily python-url, because that is somethinig I’ve run into as well. I almost feel inspired enough to start digging around the memory allocation code myself.
Almost.
Returning freed memory to the operating system is rarer than you might think. I am fairly sure that many (most?) malloc/free implementations make no attempt to use sbrk to return memory to the system.
I can see that it might be doable: you would have to pause the interpreter, move all live data in to one end of the memory map, then release the unused end to the OS. Depending on details of how the garbage collector works, this could get tricky: if the interpreter has code that uses the pointer values of objects in any way (e.g., to generate hashes for dictionaries), then moving the objects would break this code. Extensions that associate data with Python objects could also be broken when they are moved.
First, I really think, this should be clear in the doc. Because a lot of us spend a lot time on this. because, when you understand what happend, you can try to avoid this issue. (I mean write your code differently to avoid this)
I read the thread on the mailing list, last week-end. This is really interesting. Tim seems to say, this is doable, but some code will break some wrong written external module.