I've been using matplotlib with the AGG backend to generate ~10,000 plot files using PdfPages. I run this over 30 cores using python multiprocessing. This worked until a recent change.
If I run with 30 cores I have a memory issue after my recent updates (60GB of ram is exhausted and I see thrashing). If I cut that down to 24 cores I do not run into a memory issue (~45GB max usage), but I produce the segfault below at exactly 997 plots generated. I can get to about 1500 plots if I cut that down to 6 cores, and with 1 core I got that to ~3500 plots before I manually stopped it for lack of patience.
There doesn't seem to be a specific problem with any one plot, but I encounter the segfault seen below at a seemingly reproducible point depending on how many cores I run on (the more cores the sooner I encounter the issue). Memory issues seem like a likely culprit.
I recently added these constructs to my script: inset_axes(...) # and some plots using the inset_axes, I use this within subplot
ax.twiny() # to plot xticks along the top and bottom
The dmesg error seen in CentOS:
[1387989.122585] python: segfault at 7f34a7233bd0 ip 00007f3739fa8ec3 sp 00007ffedcb0aad0 error 6 in _backend_agg.so[7f3739f97000+5a000]
I'm also generating the following warning that seems to be benign:
/soe/davidparks21/anaconda2/lib/python2.7/site-packages/matplotlib/figure.py:1742: UserWarning: This figure includes Axes that are not compatible with tight_layout, so its results might be incorrect.
warnings.warn("This figure includes Axes that are not "