Quantcast

segfault in _backend_agg.so

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

segfault in _backend_agg.so

davidparks21
This post was updated on .
I've been using matplotlib with the AGG backend to generate ~10,000 plot files using PdfPages. I run this over 30 cores using python multiprocessing. This worked until a recent change.

If I run with 30 cores I have a memory issue after my recent updates (60GB of ram is exhausted and I see thrashing). If I cut that down to 24 cores I do not run into a memory issue (~45GB max usage), but I produce the segfault below at exactly 997 plots generated. I can get to about 1500 plots if I cut that down to 6 cores, and with 1 core I got that to ~3500 plots before I manually stopped it for lack of patience.

There doesn't seem to be a specific problem with any one plot, but I encounter the segfault seen below at a seemingly reproducible point depending on how many cores I run on (the more cores the sooner I encounter the issue). Memory issues seem like a likely culprit.

I recently added these constructs to my script:
  inset_axes(...)     # and some plots using the inset_axes, I use this within subplot
  ax.twiny()           # to plot xticks along the top and bottom

The dmesg error seen in CentOS:

[1387989.122585] python[24780]: segfault at 7f34a7233bd0 ip 00007f3739fa8ec3 sp 00007ffedcb0aad0 error 6 in _backend_agg.so[7f3739f97000+5a000]

I'm also generating the following warning that seems to be benign:

/soe/davidparks21/anaconda2/lib/python2.7/site-packages/matplotlib/figure.py:1742: UserWarning: This figure includes Axes that are not compatible with tight_layout, so its results might be incorrect.
  warnings.warn("This figure includes Axes that are not "


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: segfault in _backend_agg.so

davidparks21
Problem solved. On plt.subplots I had dpi=900. Removing this resolved the problem.

This line causes the segfault
fig, ax = plt.subplots(n_plots_offset, figsize=(20,3.75), sharex=False, dpi=900)

This line does not:
fig, ax = plt.subplots(n_plots_offset, figsize=(20,3.75), sharex=False)
Loading...