%matplotlib ipympl does not allow cell "re-plot" ?

[Mike-HubGit]

Describe the issue

Starting from a fresh notebook/python3 in JupyterLab 2.1.5, with 2 cells [1] and [2] :
[1] :

%matplotlib ipympl
import matplotlib.pyplot as plt

[2] :

plt.plot([1,2,3])

If I run [1] then [2], okay I see the plot.
If I run again [2], I obtain :

[<matplotlib.lines.Line2D at 0x11b33ba90>]

and cannot see the plot.
But if I update [2] like :
[2] :

%matplotlib ipympl
plt.plot([1,2,3])

Then I see the plot again. Weird as it does not work like this with %matplotlib inline or with %matplotlib notebook with Jupiter (not Lab).

Another thing surprizing, adding a "figure" :
[1] :

%matplotlib ipympl
import matplotlib.pyplot as plt

[2] :

plt.figure(1)
plt.plot([1,2,3])

[3] :

plt.figure(1)

Displays a png picture of the plot at the level of cell [3] ...

Versions

 3.7.7 (default, May  6 2020, 04:59:01) 
[Clang 4.0.1 (tags/RELEASE_401/final)]
ipympl version: 0.5.6
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : 4.7.5
ipython          : 7.16.1
ipykernel        : 5.3.2
jupyter client   : 6.1.6
jupyter lab      : 2.1.5
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.7
traitlets        : 4.3.3
Known nbextensions:
  config dir: /Users/mmyara/.jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
  config dir: /opt/miniconda3/etc/jupyter/nbconfig
    notebook section
      jupyter-matplotlib/extension  enabled 
      - Validating: OK
      plotlywidget/extension  enabled 
      - Validating: OK
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
    tree section
      ipyparallel/main  enabled 
      - Validating: OK
JupyterLab v2.1.5
Known labextensions:
   app dir: /opt/miniconda3/share/jupyter/lab
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @jupyterlab/toc v4.0.0  enabled  OK
        @krassowski/jupyterlab-lsp v1.0.0  enabled  OK
        @lckr/jupyterlab_variableinspector v0.5.1  enabled  OK
        jupyter-matplotlib v0.7.2  enabled  OK
        jupyterlab-plotly v4.8.2  enabled  OK
        plotlywidget v4.8.2  enabled  OK

[ianhi]

Hi @Mike-HubGit the behavior you describe is definitely confusing but I believe it is actually expected. Whenever a new figure is created in a cell it is automatically displayed (see updated example in #244) you can see this is if you call plt.figure() in a cell, a blank figure will show up in the output. With that context I think we can explain each of the phenomena you noted:

1:
On the very first call to plt.plot there are no active figures so you are creating one implicitly, thus the figure will show up in the output. On running cell [2] again the plt.plot command will find that there now is an active figure and so it will no create one and the cell output will simply be the last thing in the cell, in this case the object returned by plt.plot.

2:
I think what's happening here is that %matplotlib ipympl is wiping out all existing figures so then a figure shows up for the same reasons as in 1. Weird that it's different than %matplotlib notebook though.

3:
Displaying a figure as the output will show up as a png. On the first call to plt.figure(1) you create the figure with id=1 and the interactive version is displayed in the output. On the second call of plt.figure(1) you get a reference back to that figure, if it is the last line in a cell then it will be equivalent to display(plt.figure(1)) which results in a png. If you want to explicitly display the interactive figure you need to display fig.canvas. I find this a bit confusing and hard to phrase properly so here is an example that playing around with may help understanding:

Cell 1:

fig = plt.figure()
plt.plot([1,2,3])

Cell 2:

display(fig)
display(fig.canvas)

I would expect the outputs to be:
Cell 1:

• interactive plot
• [<matplotlib.lines.Line2D at 0x11b33ba90>]

Cell 2:

• png of plot
• interactive plot

hope this is helpful, feel free to ask more questions as this can definitely be a bit mystifying

[Mike-HubGit]

Thanks for your comments. However :

• what surprizes me is that invoking 'fig' does not always give the same result. It is weird to get png "sometimes" isn't it ?
• in practical situation, I write code in a cell and run it may times before it is satisfactory. Because I require a plt.figure() each time, ipympl generates a new figure and the standard limit to 20 is quickly reached. So I need to put somewhere a close('all') which is surprizing. It would be far better if, when I evaluate the code in a cell, the figures that were created in before were automatically deleted. That may avoid lots of problems. Or there is something I did not understand. Is there a sample code somewhere that is sutable for this situation (which is usual for me) ?

Thanks a lot,
Mike

[thomasaarholt]

I normally just stick plt.rcParams['figure.max_open_warning'] = 0 at the top of the cell, and then there's no warning.
Personally I think the limit of 20 figures is really low. It would be nice if the limit was raised to, say, 2000.

[Mike-HubGit]

I believed the developpers made a limit to 20 for reasons of limited resources ?

[ianhi]

. So I need to put somewhere a close('all') which is surprizing. It would be far better if, when I evaluate the code in a cell, the figures that were created in before were automatically deleted. That may avoid lots of problems. Or there is something I did not understand. Is there a sample code somewhere that is sutable for this situation (which is usual for me) ?

I often do the following:

plt.close('some-fig-name')
plt.figure('some-fig-name')
# do whatever plotting you want

or i suppose you could do:

fig = plt.figure('some-fig-name')
fig.cla()
# do whatever plotting you want

I believed the developpers made a limit to 20 for reasons of limited resources ?

The rationale for adding it is here: matplotlib/matplotlib#1919 (comment) and part of the quote is: "This could be an RC parameter so that users who do want to open 1000 figures (and have the RAM for it) can." so if you do in fact have the ram you should be able so set it to whatever you want. Also that limit was added in 2013 and I suppose that what a standard amount of ram is has increased so you probably have some wiggle room.

what surprizes me is that invoking 'fig' does not always give the same result. It is weird to get png "sometimes" isn't it ?

How do you mean by "invoking 'fig'"? whenever you display fig you will get a png

[tacaswell]

I am not clear on how you would effectively use more than ~20 figures. With notebook and all of the GUI backends when the figure get closed / is no longer visible to the users it is implicitly closed. I think ipympl does not currently close the figure when the (last) view of it is removed from the DOM. Put another way, I think that ipympl is (weakly) leaking resources, the correct fix is to stop the leak, not to remove the warning that it is leaking ;)

[Mike-HubGit]

@tacaswell : On my configuration based on Jupyter 1.3.2, if I plot a single figure in a cell and that I run it many times I reach the 20 figures limit. This is a classical process as I change elements of the cell to reach the result that I want.
The behaviour that you describe is the one that I would like, but as a beginer in the JupyterLab/Python/iPython, etc. context (I "come" from C/C++/Matlab), I have problems understanding if the actual behaviour is the expected one, making me thinking that I do not use it "the good way" (and the please inform me about the good one !), or if it is simply a bug.

[Mike-HubGit]

@ianhi :

plt.close('some-fig-name')
plt.figure('some-fig-name')
# do whatever plotting you want

Is actually what I do. However it is weird to close a figure that has never been opened. In fact : I would like to use this great environment with my students, and I search some clean message to give to them. If some behaviours are weird to me, they will be for them to, and it is a bad message about the computer as a tool for scientific work.

When I invoke "display", in most cases I obtain a widget, but sometimes a png, and sometimes both (!!) depending on in what order I write the commands.

[tacaswell]

I think this is a bug in ipympl that it is leaking the invisible, but still open figures. The use you described should work. The workarounds that @ianhi suggests above are both reasonable if a bit verbose.

The core of the issue here is that the conventions of Matplotlib (which long pre-dates jupyter) have some assumptions that the interactive figures are GUI windows baked in. To "show" a GUI window you tell the tool kit to show it, it defers to the underlying window manager, and the user sees it. When the user close the window we have un-ambigious hooks that that has happened and delete the figure from the pyplot state. This is a bit messier in the notebook as adding / removing to the DOM are more complex than popping a window up.

There is also the convention that the last thing in a cell gets auto-displayed so if you explicitly say "show this figure", it gets added to the DOM and then it also happens to be the last thing in the cell so it also gets displayed. There is some effort at doing effectively "display refernce" counting to try to suppress that, but if you are seeing two copies then it is not being suppressed enough.

There is also the issue that rich repr of the figure (png) and the figure.canvas (widget) are different .

[Mike-HubGit]

@tacaswell : I understand. However : the "notebook" mode if jupyter (without lab) worked well and I do not remember the problems you mention.
This issue is known by the developers ? Do you know if they plan to solve it ?
Thanks a lot,
Mike

[ianhi]

@Mike-HubGit are you running the cell multiple times to adjust parameters of the lines you are plotting, or in order to adjust styling and such? If you are aiming for the former usecase then I'd recommend using the sliders provided by ipywidgets to continuously update the lines as you tune the parameters by hand.

This issue is known by the developers ? Do you know if they plan to solve it ?

I think this is a longstanding issue, and unfortunately a tricky one to solve. See: #4

Also this issue of the figure warning has also been discussed here: #64

[Mike-HubGit]

@ianhi : for many reasons ! debugging, parameters, styling.
Ok I understand these problems. Thanks for this discussion, hope it will be solved ... a day ...