Skip to content

python3Packages.dask-yarn: init at 0.9 - #165226

Merged
SuperSandro2000 merged 2 commits into
NixOS:masterfrom
illustris:dask-yarn
Apr 28, 2022
Merged

python3Packages.dask-yarn: init at 0.9#165226
SuperSandro2000 merged 2 commits into
NixOS:masterfrom
illustris:dask-yarn

Conversation

@illustris

@illustris illustris commented Mar 22, 2022

Copy link
Copy Markdown
Contributor
Description of changes

homepage: https://yarn.dask.org/

Dask-Yarn deploys Dask on YARN clusters, such as are found in traditional Hadoop installations. Dask-Yarn provides an easy interface to quickly start, scale, and stop Dask clusters natively from Python.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions Bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Mar 22, 2022
@ofborg ofborg Bot added 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 22, 2022
Comment thread pkgs/development/python-modules/skein/skeinjar.nix Outdated
Comment thread pkgs/development/python-modules/skein/skeinjar.nix Outdated
Comment thread pkgs/development/python-modules/skein/skeinjar.nix Outdated
Comment thread pkgs/development/python-modules/skein/skeinjar.nix Outdated
Comment thread pkgs/development/python-modules/skein/default.nix Outdated
Comment thread pkgs/development/python-modules/dask-yarn/default.nix Outdated
Comment thread pkgs/development/python-modules/dask-yarn/default.nix Outdated
@illustris
illustris force-pushed the dask-yarn branch 2 times, most recently from e76e6d9 to 8fc621e Compare March 23, 2022 04:06
Comment thread pkgs/development/python-modules/dask-yarn/default.nix Outdated
Comment thread pkgs/development/python-modules/dask-yarn/default.nix Outdated
Comment thread pkgs/development/python-modules/skein/default.nix Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pytest goes into checkInputs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried that before, didn't work.

FAILED (errors=13)
Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>
error: builder for '/nix/store/cp7hgpfnqww49xz99227v1x9dan0j3ml-python3.9-skein-0.8.1.drv' failed with exit code 1;
       last 10 log lines:
       >     import pytest
       > ModuleNotFoundError: No module named 'pytest'
       >
       >
       > ----------------------------------------------------------------------
       > Ran 13 tests in 0.001s
       >
       > FAILED (errors=13)
       > Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>
       > error: Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>
       For full logs, run 'nix log /nix/store/cp7hgpfnqww49xz99227v1x9dan0j3ml-python3.9-skein-0.8.1.drv'.

skein appears to be calling pytest as part of its build steps

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide the full log? I am not sure how this is loaded in the build step. Is it already doing tests there?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing setuptools-check-hook
Using setuptoolsCheckPhase
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/b2wmn6dcz4v5qmcy65n82l7wy378qbxy-skein-0.8.1.tar.gz
source root is skein-0.8.1
setting SOURCE_DATE_EPOCH to timestamp 1614705207 of file skein-0.8.1/skein/_version.py
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_proto
running build_py
creating build
creating build/lib
creating build/lib/skein
copying skein/objects.py -> build/lib/skein
copying skein/exceptions.py -> build/lib/skein
copying skein/model.py -> build/lib/skein
copying skein/cli.py -> build/lib/skein
copying skein/core.py -> build/lib/skein
copying skein/__init__.py -> build/lib/skein
copying skein/tornado.py -> build/lib/skein
copying skein/kv.py -> build/lib/skein
copying skein/utils.py -> build/lib/skein
copying skein/_version.py -> build/lib/skein
copying skein/ui.py -> build/lib/skein
creating build/lib/skein/proto
copying skein/proto/__init__.py -> build/lib/skein/proto
copying skein/proto/skein_pb2_grpc.py -> build/lib/skein/proto
copying skein/proto/skein_pb2.py -> build/lib/skein/proto
creating build/lib/skein/recipes
copying skein/recipes/ipython_kernel.py -> build/lib/skein/recipes
copying skein/recipes/__init__.py -> build/lib/skein/recipes
copying skein/recipes/jupyter_notebook.py -> build/lib/skein/recipes
creating build/lib/skein/java
copying skein/java/skein.jar -> build/lib/skein/java
UPDATING build/lib/skein/_version.py
set build/lib/skein/_version.py to '0.8.1'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/objects.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/core.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/kv.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/cli.py -> build/bdist.linux-x86_64/wheel/skein
creating build/bdist.linux-x86_64/wheel/skein/java
copying build/lib/skein/java/skein.jar -> build/bdist.linux-x86_64/wheel/skein/java
creating build/bdist.linux-x86_64/wheel/skein/proto
copying build/lib/skein/proto/skein_pb2_grpc.py -> build/bdist.linux-x86_64/wheel/skein/proto
copying build/lib/skein/proto/skein_pb2.py -> build/bdist.linux-x86_64/wheel/skein/proto
copying build/lib/skein/proto/__init__.py -> build/bdist.linux-x86_64/wheel/skein/proto
copying build/lib/skein/model.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/ui.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/tornado.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/utils.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/__init__.py -> build/bdist.linux-x86_64/wheel/skein
creating build/bdist.linux-x86_64/wheel/skein/recipes
copying build/lib/skein/recipes/__init__.py -> build/bdist.linux-x86_64/wheel/skein/recipes
copying build/lib/skein/recipes/jupyter_notebook.py -> build/bdist.linux-x86_64/wheel/skein/recipes
copying build/lib/skein/recipes/ipython_kernel.py -> build/bdist.linux-x86_64/wheel/skein/recipes
copying build/lib/skein/_version.py -> build/bdist.linux-x86_64/wheel/skein
copying build/lib/skein/exceptions.py -> build/bdist.linux-x86_64/wheel/skein
running install_egg_info
running egg_info
writing skein.egg-info/PKG-INFO
writing dependency_links to skein.egg-info/dependency_links.txt
writing entry points to skein.egg-info/entry_points.txt
writing requirements to skein.egg-info/requires.txt
writing top-level names to skein.egg-info/top_level.txt
reading manifest file 'skein.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'java/target'
warning: no previously-included files found matching 'java/dependency-reduced-pom.xml'
adding license file 'LICENSE'
writing manifest file 'skein.egg-info/SOURCES.txt'
Copying skein.egg-info to build/bdist.linux-x86_64/wheel/skein-0.8.1-py3.9.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/skein-0.8.1.dist-info/WHEEL
creating 'dist/skein-0.8.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'skein/__init__.py'
adding 'skein/_version.py'
adding 'skein/cli.py'
adding 'skein/core.py'
adding 'skein/exceptions.py'
adding 'skein/kv.py'
adding 'skein/model.py'
adding 'skein/objects.py'
adding 'skein/tornado.py'
adding 'skein/ui.py'
adding 'skein/utils.py'
adding 'skein/java/skein.jar'
adding 'skein/proto/__init__.py'
adding 'skein/proto/skein_pb2.py'
adding 'skein/proto/skein_pb2_grpc.py'
adding 'skein/recipes/__init__.py'
adding 'skein/recipes/ipython_kernel.py'
adding 'skein/recipes/jupyter_notebook.py'
adding 'skein-0.8.1.dist-info/LICENSE'
adding 'skein-0.8.1.dist-info/METADATA'
adding 'skein-0.8.1.dist-info/WHEEL'
adding 'skein-0.8.1.dist-info/entry_points.txt'
adding 'skein-0.8.1.dist-info/top_level.txt'
adding 'skein-0.8.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsBuildPhase
@nix { "action": "setPhase", "phase": "installPhase" }
installing
Executing pipInstallPhase
/build/skein-0.8.1/dist /build/skein-0.8.1
Processing ./skein-0.8.1-py3-none-any.whl
Requirement already satisfied: cryptography in /nix/store/1ak7jkgjyyqxwy13cm2qzfvdyy8d47nc-python3.9-cryptography-36.0.0/lib/python3.9/site-packages (from skein==0.8.1) (36.0.0)
Requirement already satisfied: pyyaml in /nix/store/va4j4930kj0svadrq01pvv83k3rvbl59-python3.9-pyyaml-6.0/lib/python3.9/site-packages (from skein==0.8.1) (6.0)
Requirement already satisfied: protobuf>=3.5.0 in /nix/store/w51jrjasqy1dh71bvwcd4jdaf7yy63wk-python3.9-protobuf-3.19.3/lib/python3.9/site-packages (from skein==0.8.1) (3.19.3)
Requirement already satisfied: grpcio>=1.11.0 in /nix/store/682808rzfh3lkvhl48ppm9kx2z7hy00c-python3.9-grpcio-1.43.0/lib/python3.9/site-packages (from skein==0.8.1) (1.43.0)
Requirement already satisfied: six>=1.5.2 in /nix/store/5cqp0x613vxqlad25w18fmzcyi0p7lzv-python3.9-six-1.16.0/lib/python3.9/site-packages (from grpcio>=1.11.0->skein==0.8.1) (1.16.0)
Requirement already satisfied: cffi>=1.12 in /nix/store/8nhm05fvzscamaqmcpqfkqna6lwdb4m1-python3.9-cffi-1.15.0/lib/python3.9/site-packages (from cryptography->skein==0.8.1) (1.15.0)
Requirement already satisfied: pycparser in /nix/store/ykg29kij732bj9vic5vsw993s9wq1686-python3.9-pycparser-2.21/lib/python3.9/site-packages (from cffi>=1.12->cryptography->skein==0.8.1) (2.21)
Installing collected packages: skein
Successfully installed skein-0.8.1
/build/skein-0.8.1
Finished executing pipInstallPhase
@nix { "action": "setPhase", "phase": "fixupPhase" }
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1
strip is /nix/store/jz10kjr0lnhynwxnrzv4kcwbjs5s0ri8-gcc-wrapper-10.3.0/bin/strip
stripping (with command strip and flags -S) in /nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1/lib  /nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1/bin 
patching script interpreter paths in /nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1
checking for references to /build/ in /nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1...
Rewriting #!/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/bin/python3.9 to #!/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10
wrapping `/nix/store/8vy26qphxs09p0626zanspnn0n33rq9p-python3.9-skein-0.8.1/bin/skein'...
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
@nix { "action": "setPhase", "phase": "installCheckPhase" }
running install tests
no Makefile or custom installCheckPhase, doing nothing
@nix { "action": "setPhase", "phase": "pythonCatchConflictsPhase" }
pythonCatchConflictsPhase
@nix { "action": "setPhase", "phase": "pythonRemoveBinBytecodePhase" }
pythonRemoveBinBytecodePhase
@nix { "action": "setPhase", "phase": "pythonImportsCheckPhase" }
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
Check whether the following modules can be imported: skein
@nix { "action": "setPhase", "phase": "setuptoolsCheckPhase" }
setuptoolsCheckPhase
Executing setuptoolsCheckPhase
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing skein.egg-info/PKG-INFO
writing dependency_links to skein.egg-info/dependency_links.txt
writing entry points to skein.egg-info/entry_points.txt
writing requirements to skein.egg-info/requires.txt
writing top-level names to skein.egg-info/top_level.txt
reading manifest file 'skein.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'java/target'
warning: no previously-included files found matching 'java/dependency-reduced-pom.xml'
adding license file 'LICENSE'
writing manifest file 'skein.egg-info/SOURCES.txt'
running build_ext
test_kv (unittest.loader._FailedTest) ... ERROR
test_model (unittest.loader._FailedTest) ... ERROR
test_ui (unittest.loader._FailedTest) ... ERROR
test_tornado (unittest.loader._FailedTest) ... ERROR
test_cli (unittest.loader._FailedTest) ... ERROR
test_core (unittest.loader._FailedTest) ... ERROR
conftest (unittest.loader._FailedTest) ... ERROR
skein.test.test_cli (unittest.loader._FailedTest) ... ERROR
skein.test.test_core (unittest.loader._FailedTest) ... ERROR
skein.test.test_kv (unittest.loader._FailedTest) ... ERROR
skein.test.test_model (unittest.loader._FailedTest) ... ERROR
skein.test.test_tornado (unittest.loader._FailedTest) ... ERROR
skein.test.test_ui (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: test_kv (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_kv
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_kv.py", line 10, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: test_model (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_model
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_model.py", line 7, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: test_ui (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_ui
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_ui.py", line 1, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: test_tornado (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_tornado
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_tornado.py", line 4, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: test_cli (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_cli
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_cli.py", line 9, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: test_core (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_core
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/test_core.py", line 11, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: conftest (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: conftest
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/skein-0.8.1/skein/test/conftest.py", line 6, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_cli (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_cli
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_cli.py", line 9, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_core (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_core
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_core.py", line 11, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_kv (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_kv
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_kv.py", line 10, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_model (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_model
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_model.py", line 7, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_tornado (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_tornado
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_tornado.py", line 4, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


======================================================================
ERROR: skein.test.test_ui (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: skein.test.test_ui
Traceback (most recent call last):
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/skein-0.8.1/skein/test/test_ui.py", line 1, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


----------------------------------------------------------------------
Ran 13 tests in 0.001s

FAILED (errors=13)
Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=13 errors=13 failures=0>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats already in checkPhase. Please try adding pytestCheckHook to checkInputs and then it should work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember now, this is how I tried to do it first. If pytest is run in checkPhase, it will get stuck.

diff:

diff --git a/pkgs/development/python-modules/skein/default.nix b/pkgs/development/python-modules/skein/default.nix
index 59b47dba120..b0378c7a6d7 100644
--- a/pkgs/development/python-modules/skein/default.nix
+++ b/pkgs/development/python-modules/skein/default.nix
@@ -10,6 +10,7 @@
 , pytest
 , grpcio-tools
 , hadoop
+, pytestCheckHook
 }:

 buildPythonPackage rec {
@@ -26,7 +27,7 @@ buildPythonPackage rec {
   disabled = isPy27;

   propagatedBuildInputs = [ cryptography grpcio pyyaml ];
-  buildInputs = [ pytest jupyter grpcio-tools ];
+  buildInputs = [ jupyter grpcio-tools ];

   preBuild = ''
     # Ensure skein.jar exists skips the maven build in setup.py
@@ -39,6 +40,8 @@ buildPythonPackage rec {
       --replace "else 'java'" "else '${hadoop.jdk}/bin/java'"
   '';

+  checkInputs = [ pytestCheckHook ];
+
   pythonImportsCheck = [ "skein" ];

   meta = with lib; {

build log:

.
.
.
Finished executing setuptoolsBuildPhase
installing
Executing pipInstallPhase
/build/skein-0.8.1/dist /build/skein-0.8.1
Processing ./skein-0.8.1-py3-none-any.whl
Requirement already satisfied: cryptography in /nix/store/1ak7jkgjyyqxwy13cm2qzfvdyy8d47nc-python3.9-cryptography-36.0.0/lib/python3.9/site-packages (from skein==0.8.1) (36.0.0)
Requirement already satisfied: pyyaml in /nix/store/va4j4930kj0svadrq01pvv83k3rvbl59-python3.9-pyyaml-6.0/lib/python3.9/site-packages (from skein==0.8.1) (6.0)
Requirement already satisfied: protobuf>=3.5.0 in /nix/store/w51jrjasqy1dh71bvwcd4jdaf7yy63wk-python3.9-protobuf-3.19.3/lib/python3.9/site-packages (from skein==0.8.1) (3.19.3)
Requirement already satisfied: grpcio>=1.11.0 in /nix/store/682808rzfh3lkvhl48ppm9kx2z7hy00c-python3.9-grpcio-1.43.0/lib/python3.9/site-packages (from skein==0.8.1) (1.43.0)
Requirement already satisfied: six>=1.5.2 in /nix/store/5cqp0x613vxqlad25w18fmzcyi0p7lzv-python3.9-six-1.16.0/lib/python3.9/site-packages (from grpcio>=1.11.0->skein==0.8.1) (1.16.0)
Requirement already satisfied: cffi>=1.12 in /nix/store/8nhm05fvzscamaqmcpqfkqna6lwdb4m1-python3.9-cffi-1.15.0/lib/python3.9/site-packages (from cryptography->skein==0.8.1) (1.15.0)
Requirement already satisfied: pycparser in /nix/store/ykg29kij732bj9vic5vsw993s9wq1686-python3.9-pycparser-2.21/lib/python3.9/site-packages (from cffi>=1.12->cryptography->skein==0.8.1) (2.21)
Installing collected packages: skein
Successfully installed skein-0.8.1
/build/skein-0.8.1
Finished executing pipInstallPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1
strip is /nix/store/jz10kjr0lnhynwxnrzv4kcwbjs5s0ri8-gcc-wrapper-10.3.0/bin/strip
stripping (with command strip and flags -S) in /nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1/lib  /nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1/bin
patching script interpreter paths in /nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1
checking for references to /build/ in /nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1...
Rewriting #!/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/bin/python3.9 to #!/nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10
wrapping `/nix/store/ihls0xl5xzaq9p0w45dy5pxbw88ch3zw-python3.9-skein-0.8.1/bin/skein'...
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
running install tests
no Makefile or custom installCheckPhase, doing nothing
pythonCatchConflictsPhase
pythonRemoveBinBytecodePhase
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
Check whether the following modules can be imported: skein
pytestCheckPhase
Executing pytestCheckPhase
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /build/skein-0.8.1
collected 213 items

It just stays there

nix-top:

:: (nixbld1) → /tmp/nix-build-python3.9-skein-0.8.1.drv-0
  UID     PID    PPID STIME     TIME COMMAND
30001 3621785 3619506 21:15 00:00:00 bash -e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
30001 3622133 3621785 21:15 00:00:01 /nix/store/hb1lzaisgx2m9n29hqhh6yp6hasplq1v-python3-3.9.10/bin/python3.9 -m pytest
30001 3622448 3622133 21:15 00:00:04 /nix/store/fq64f8garc0h9936v63xvsjahr79j9hc-openjdk-headless-11.0.12+7/bin/java -Dskein.log.level=INFO com.anaconda.skein.Driver --jar /build/skein-0.8.1/skein/java/skein.jar

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay... weird.

@jonringer can you help? I am a bit out of ideas without actually debugging this.

@illustris
illustris force-pushed the dask-yarn branch 2 times, most recently from 361fddc to b8d59e0 Compare March 28, 2022 20:44
illustris and others added 2 commits April 18, 2022 13:40
- Use prebuilt jar to get around maven builds being non-reproducible
- Remove unnecessary propagated dependencies
- Inherit java version from hadoop
Co-authored-by: ankeshjha999 <ankeshjha999@gmail.com>
@illustris

Copy link
Copy Markdown
Contributor Author

The reason setuptoolsCheck worked but pytest didn't is that setuptools was not actually running any tests. Many of the tests in skein require network connectivity to work. I've disabled those tests and only left tests that can be done offline.

@nixos-discourse

Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/815

maintainers = with maintainers; [ alexbiehl illustris ];
# https://github.com/NixOS/nixpkgs/issues/48663#issuecomment-1083031627
# replace with https://github.com/NixOS/nixpkgs/pull/140325 once it is merged
broken = lib.traceIf isPy27 "${pname} not supported on ${python.executable}" isPy27;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be disabled = isPy27; one level up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in the issue linked there, disabled can't be overridden. This is a treewide issue that affect multiple python packages (my last run-in with this was ipython). Until #140325 gets merged, this is the best way to mark a package as unsupported on a python version without completely blocking someone from overriding/patching it to work in their own derivation.

@SuperSandro2000
SuperSandro2000 merged commit b49cd9a into NixOS:master Apr 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants