Quantcast

memory usage with repeated imshow

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

memory usage with repeated imshow

Tom Dimiduk
I am using matplotlib pylab in association with ipython -pylab to show
many large (~2000x2000 or larger) images.  Each time I show another
image it consumes more memory until eventually exhausting all system
memory and making my whole system unresponsive.

The easiest way to replicate this behaviour is with
a = ones((3333,3333))
imshow(a)

optionally

close()

and then

imshow(a)

again.  I am using ipython .10.1 and matplotlib 0.99.3.  Is there
something I should be doing differently to avoid this problem?  Is it
fixed in a later version?

Thanks,
Tom

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory usage with repeated imshow

Robert Abiad
Tom,

I just went through this, though with version 1.01 of mpl, so it may be different.  You can read the
very long thread at:

http://www.mail-archive.com/matplotlib-users@.../msg20031.html

Those who maintain mpl don't think there is a memory leak. What I found was that imshow() does
consume a lot of memory (now fixed in the development version) and that the first 2 or so uses build
on each other, but after that it levels off giving back memory after close().  There is a
discrepancy between what python reports it's using and what the OS reports (I had 500MB from the OS,
but only 150MB from python).  There is a chance that ipython is caching your results (try ipython
-pylab -cs 0), but when I ran without ipython, python still had a large portion of memory.

-robert

On 2/9/2011 3:52 PM, Tom Dimiduk wrote:

> I am using matplotlib pylab in association with ipython -pylab to show
> many large (~2000x2000 or larger) images.  Each time I show another
> image it consumes more memory until eventually exhausting all system
> memory and making my whole system unresponsive.
>
> The easiest way to replicate this behaviour is with
> a = ones((3333,3333))
> imshow(a)
>
> optionally
>
> close()
>
> and then
>
> imshow(a)
>
> again.  I am using ipython .10.1 and matplotlib 0.99.3.  Is there
> something I should be doing differently to avoid this problem?  Is it
> fixed in a later version?
>
> Thanks,
> Tom
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Matplotlib-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory usage with repeated imshow

Tom Dimiduk
Thank you for your help.  I upgraded to the latest development version,
and as you said, memory use dropped a ton.  I will have to test more to
confirm that the problem is completely gone, but this appears to bring
memory usage down to something quite manageable (at least on my 8gb box
...).

Tom

On 02/09/2011 07:30 PM, Robert Abiad wrote:

> Tom,
>
> I just went through this, though with version 1.01 of mpl, so it may be different.  You can read the
> very long thread at:
>
> http://www.mail-archive.com/matplotlib-users@.../msg20031.html
>
> Those who maintain mpl don't think there is a memory leak. What I found was that imshow() does
> consume a lot of memory (now fixed in the development version) and that the first 2 or so uses build
> on each other, but after that it levels off giving back memory after close().  There is a
> discrepancy between what python reports it's using and what the OS reports (I had 500MB from the OS,
> but only 150MB from python).  There is a chance that ipython is caching your results (try ipython
> -pylab -cs 0), but when I ran without ipython, python still had a large portion of memory.
>
> -robert
>
> On 2/9/2011 3:52 PM, Tom Dimiduk wrote:
>> I am using matplotlib pylab in association with ipython -pylab to show
>> many large (~2000x2000 or larger) images.  Each time I show another
>> image it consumes more memory until eventually exhausting all system
>> memory and making my whole system unresponsive.
>>
>> The easiest way to replicate this behaviour is with
>> a = ones((3333,3333))
>> imshow(a)
>>
>> optionally
>>
>> close()
>>
>> and then
>>
>> imshow(a)
>>
>> again.  I am using ipython .10.1 and matplotlib 0.99.3.  Is there
>> something I should be doing differently to avoid this problem?  Is it
>> fixed in a later version?
>>
>> Thanks,
>> Tom
>>
>> ------------------------------------------------------------------------------
>> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
>> Pinpoint memory and threading errors before they happen.
>> Find and fix more than 250 security defects in the development cycle.
>> Locate bottlenecks in serial and parallel code that limit performance.
>> http://p.sf.net/sfu/intel-dev2devfeb
>> _______________________________________________
>> Matplotlib-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Matplotlib-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Loading...