[Matplotlib-devel] our tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[Matplotlib-devel] our tests

Eric Firing-2
Does anyone know why:

1) In our Travis suite, the USE_PYTEST run takes much longer than the
others.

2) On master (approximately), when I try to run a single function with
py.test, it isn't found.  Example:

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog
=============================================== test session starts
================================================
platform darwin -- Python 3.5.1, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
rootdir: /Users/efiring/work/programs/py/mpl/matplotlib, inifile: pytest.ini
collecting 530 items
=========================================== no tests ran in 0.39 seconds
===========================================
ERROR: not found:
/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog
(no name
'/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog'
in any of [<Module 'lib/matplotlib/tests/test_axes.py'>])

3) Also on master, I get errors with tests.py.  Example:

python tests.py lib/matplotlib/tests/test_axes.py

[snip]

======================================================================
ERROR: matplotlib.tests.test_axes.test_errorbar_inputs_shotgun
----------------------------------------------------------------------
Traceback (most recent call last):
   File
"/Users/efiring/anaconda/envs/test/lib/python3.5/site-packages/nose/case.py",
line 198, in runTest
     self.test(*self.arg)
   File
"/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/testing/decorators.py",
line 150, in wrapped_callable
     func(*args, **kwargs)
TypeError: test_errorbar_inputs_shotgun() missing 1 required positional
argument: 'kwargs'

======================================================================
ERROR: matplotlib.tests.test_axes.test_remove_shared_axes
----------------------------------------------------------------------
Traceback (most recent call last):
   File
"/Users/efiring/anaconda/envs/test/lib/python3.5/site-packages/nose/case.py",
line 198, in runTest
     self.test(*self.arg)
   File
"/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/testing/decorators.py",
line 150, in wrapped_callable
     func(*args, **kwargs)
TypeError: test_remove_shared_axes() missing 2 required positional
arguments: 'shared_axes_generator' and 'shared_axis_remover'

----------------------------------------------------------------------
Ran 437 tests in 142.437s

FAILED (SKIP=1, errors=2)
Exception ignored in: <generator object
TestLoader.loadTestsFromGenerator.<locals>.generate at 0x108f40ca8>
RuntimeError: generator ignored GeneratorExit

Eric

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: our tests

Nelle Varoquaux
Does anyone know why:

1) In our Travis suite, the USE_PYTEST run takes much longer than the others.

That's an easy one: the tests that are run with pytest are not run with nose.  Now that we've migrated most of the tests to pytests, the nose environment almost doesn't run any tests anymore.
 
2) On master (approximately), when I try to run a single function with py.test, it isn't found.  Example:

I can't help you on that one, but I'd like to know as well.
I usually run all the tests in a single module.
 

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog
=============================================== test session starts ================================================
platform darwin -- Python 3.5.1, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
rootdir: /Users/efiring/work/programs/py/mpl/matplotlib, inifile: pytest.ini
collecting 530 items
=========================================== no tests ran in 0.39 seconds ===========================================
ERROR: not found: /Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog
(no name '/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog' in any of [<Module 'lib/matplotlib/tests/test_axes.py'>])

3) Also on master, I get errors with tests.py.  Example:

python tests.py lib/matplotlib/tests/test_axes.py

[snip]

======================================================================
ERROR: matplotlib.tests.test_axes.test_errorbar_inputs_shotgun
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/efiring/anaconda/envs/test/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/testing/decorators.py", line 150, in wrapped_callable
    func(*args, **kwargs)
TypeError: test_errorbar_inputs_shotgun() missing 1 required positional argument: 'kwargs'

======================================================================
ERROR: matplotlib.tests.test_axes.test_remove_shared_axes
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/efiring/anaconda/envs/test/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/testing/decorators.py", line 150, in wrapped_callable
    func(*args, **kwargs)
TypeError: test_remove_shared_axes() missing 2 required positional arguments: 'shared_axes_generator' and 'shared_axis_remover'

----------------------------------------------------------------------
Ran 437 tests in 142.437s

FAILED (SKIP=1, errors=2)
Exception ignored in: <generator object TestLoader.loadTestsFromGenerator.<locals>.generate at 0x108f40ca8>
RuntimeError: generator ignored GeneratorExit

Eric

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: our tests

Nathaniel Smith
In reply to this post by Eric Firing-2
On Jan 28, 2017 8:06 AM, "Eric Firing" <[hidden email]> wrote:
Does anyone know why:

1) In our Travis suite, the USE_PYTEST run takes much longer than the others.

2) On master (approximately), when I try to run a single function with py.test, it isn't found.  Example:

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog

I've never figured out the syntax for naming a single test -- I think it might vary depending on things like whether your test directories are packages? Pytest pulls configuration from the files and directories around each test, so it kind of needs to "see" the whole test tree.

But, I did (after an embarrassingly long time) discover the fabulous -k switch. The idea is you run pytest normally so it can see your whole test suite, but then "-k hist_steplog" tells it to ignore any tests whose name doesn't have that as a substring.

-n

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: our tests

Antony Lee
FWIW there is an open issue (generally approved by the pytest devs) to simplify selection of a single test: https://github.com/pytest-dev/pytest/issues/2193

2017-01-28 11:49 GMT-08:00 Nathaniel Smith <[hidden email]>:
On Jan 28, 2017 8:06 AM, "Eric Firing" <[hidden email]> wrote:
Does anyone know why:

1) In our Travis suite, the USE_PYTEST run takes much longer than the others.

2) On master (approximately), when I try to run a single function with py.test, it isn't found.  Example:

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog

I've never figured out the syntax for naming a single test -- I think it might vary depending on things like whether your test directories are packages? Pytest pulls configuration from the files and directories around each test, so it kind of needs to "see" the whole test tree.

But, I did (after an embarrassingly long time) discover the fabulous -k switch. The idea is you run pytest normally so it can see your whole test suite, but then "-k hist_steplog" tells it to ignore any tests whose name doesn't have that as a substring.

-n

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel



_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

[Matplotlib-devel] Fwd: our tests

Elliott Sales de Andrade
In reply to this post by Eric Firing-2
Hi Eric,

Sorry if you receive this twice.

On 24 November 2016 at 18:49, Eric Firing <[hidden email]> wrote:
Does anyone know why:

1) In our Travis suite, the USE_PYTEST run takes much longer than the others.


Nelle has answered this one already.
 
2) On master (approximately), when I try to run a single function with py.test, it isn't found.  Example:

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog
=============================================== test session starts ================================================
platform darwin -- Python 3.5.1, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
rootdir: /Users/efiring/work/programs/py/mpl/matplotlib, inifile: pytest.ini
collecting 530 items
=========================================== no tests ran in 0.39 seconds ===========================================
ERROR: not found: /Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog
(no name '/Users/efiring/work/programs/py/mpl/matplotlib/lib/matplotlib/tests/test_axes.py::test_hist_steplog' in any of [<Module 'lib/matplotlib/tests/test_axes.py'>])


This works for me; I think you need to update your version of pytest:

py.test lib/matplotlib/tests/test_axes.py::test_hist_steplog
============================= test session starts =============================
platform linux -- Python 3.5.2, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
rootdir: /home/elliott/code/matplotlib, inifile: pytest.ini
collected 505 items

lib/matplotlib/tests/test_axes.py ...

========================== 3 passed in 3.78 seconds ===========================

Note, there are 3 tests because of the 3 extensions that are checked by the image_comparison decorator.
 
3) Also on master, I get errors with tests.py.  Example:

python tests.py lib/matplotlib/tests/test_axes.py

[snip]



This will not work at the moment. tests.py continues to use nose, so it will not run pytest-based files. The list of files that only work with nose is very short right now. As soon as my other PR is merged, I will open a PR that should make this work again (there are conflicts).

--
Elliott



--
Elliott

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel