Asyncio, Tk, and matplotlib

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
jni
Reply | Threaded
Open this post in threaded view
|

Asyncio, Tk, and matplotlib

jni
Hi everyone,

I’m new to both asynchronous programming and to GUI programming so please forgive any newbie mistakes in the below code.

I’m developing a simple GUI app that takes some parameters from the user, then when the user clicks a button, launches some long-running task, *which includes making and saving a matplotlib plot*. When using the standard Tk `app.mainloop()`, this causes the application to become non-responsive while the long computation is running.

So I figured I’d use this newfangled asyncio thnigy that everyone is talking about. =P I settled on modelling it after Guido’s own tkcrawl.py [1]_, which periodically “manually" calls tkinter’s `update()` within the asyncio event loop. Unfortunately, when matplotlib is invoked in the asynchronous task, it crashes tkinter with the error:

RuntimeError: main thread is not in main loop

Apparently calling tk functions from anything other than the main thread is a big no-no.

But, this is where I’m stuck: how do I asynchronously launch a long-running task that includes matplotlib plotting? Any ideas about how I can tweak my design so my app is responsive but tasks include making big plots?

I’ve included sample code below. By swapping out “fasync” for “fsync”, you too can experience the frustration of a beach-balled or greyed out app — but at least the app works! ;)

Thanks!

Juan.

.. [1] https://bugs.python.org/file43873/tkcrawl.py


Minimal code:

import asyncio
import matplotlib
matplotlib.use('TkAgg')
import tkinter as tk
from tkinter import ttk
from skimage._shared._tempfile import temporary_file
import numpy as np


@asyncio.coroutine
def _async(func, *args):
    loop = asyncio.get_event_loop()
    return (yield from loop.run_in_executor(None, func, *args))


STANDARD_MARGIN = (3, 3, 12, 12)


def long_computation():
    import time
    time.sleep(4)
    import matplotlib.pyplot as plt
    fig, ax = plt.subplots()
    ax.imshow(np.random.rand(500, 500))
    with temporary_file(suffix='.png') as fname:
        fig.savefig(fname)


class MainWindow(tk.Tk):
    def __init__(self):
        super().__init__()
        self.title('I gonna die')
        main = ttk.Frame(master=self, padding=STANDARD_MARGIN)
        main.grid(row=0, column=0, sticky='nsew')
        fsync = long_computation
        fasync = lambda: asyncio.ensure_future(_async(long_computation))
        button = ttk.Button(master=main, padding=STANDARD_MARGIN,
                            text='Run stuff',
                            command=fasync)
        button.grid(row=0, column=0)
        main.pack()


def tk_update(loop, app):
    try:
        app.update()
    except tk.TclError as e:
        loop.stop()
        return
    loop.call_later(.01, tk_update, loop, app)


def main():
    loop = asyncio.get_event_loop()
    app = MainWindow()
    #app.mainloop()
    tk_update(loop, app)
    loop.run_forever()


if __name__ == '__main__':
    main()


_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-users
Reply | Threaded
Open this post in threaded view
|

Re: Asyncio, Tk, and matplotlib

Ludwig Schwardt-2
Hi Juan,

Funny that you should mention it... I was busy with the exact same thing today :-)

It is as you said: the matplotlib GUI has to run on the main thread. The trick seems to be to start the asyncio loop in a second background thread and communicate with the GUI running on the main thread via a queue. I found this StackOverflow answer very helpful. 

As an aside, I eventually ditched asyncio for an even simpler threading + queue solution (being stuck in Python 2.7...).

Cheers,

Ludwig

P.P.S. I have an old matplotlib GUI app that used to disable the button during the long-running task (run from the on_clicked callback) to indicate when the GUI becomes available again. This behaviour does not seem to work anymore on modern matplotlib, hence my need to investigate background threads :-)


_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-users
Reply | Threaded
Open this post in threaded view
|

Re: Asyncio, Tk, and matplotlib

tcaswell
Sorry I don't have time to write out a longer answer (with, you know, working code), but a of couple of rough thoughts from recent experience.

 - all GUI stuff must happen on the main thread (if you do threads), but for some backends `draw_idle()` maybe thread safe.
 - for asyncio no blocking part of the task should be slower than 1/10 a second (that is the time between subsequent calls to `yield from`/`yield`/`await`).  Anything slower and you will need to push it out of the main thread/process.  It brings a bunch of complexity, but I would look at something like dask.  The futures look to have a `add_done_callback` method which you can use to bridge to an asyncio.Event that you can await on, (although you might run into some tornado vs asyncio issues).  Looks like someone has already done the work of hiding multiprocess behind asyncio (https://github.com/dano/aioprocessing). Would not go with threads unless you have a lot of gil releasing code.
 - integrating asyncio and GUIs are about the same problem as integrating GUIs and the command line, you have two infinite loops that both want to run the show (and block while waiting for the slow human).  I have been using https://github.com/NSLS-II/bluesky/blob/master/bluesky/utils.py#L684 to good effect at my day-job for keeping figures alive under asyncio, but we are mostly waiting for (motion control related) network / motion.  You install a self-perpetuating 'call_later' on to the asyncio event loop that drains the GUI events (which lets all of their callbacks run and re-draws the figure).
 - A super embarrassing (but functional) qt example is where I use qt threads + ipyparallel is https://github.com/tacaswell/leidenfrost/blob/master/leidenfrost/gui/proc_gui.py In this case I let the GUI event loop run the show.

Tom



On Thu, Feb 23, 2017 at 3:45 PM Ludwig Schwardt <[hidden email]> wrote:
Hi Juan,

Funny that you should mention it... I was busy with the exact same thing today :-)

It is as you said: the matplotlib GUI has to run on the main thread. The trick seems to be to start the asyncio loop in a second background thread and communicate with the GUI running on the main thread via a queue. I found this StackOverflow answer very helpful. 

As an aside, I eventually ditched asyncio for an even simpler threading + queue solution (being stuck in Python 2.7...).

Cheers,

Ludwig

P.P.S. I have an old matplotlib GUI app that used to disable the button during the long-running task (run from the on_clicked callback) to indicate when the GUI becomes available again. This behaviour does not seem to work anymore on modern matplotlib, hence my need to investigate background threads :-)

_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-users

_______________________________________________
Matplotlib-users mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-users