GNU bug report logs - #57136
Snakemake cannot execute remote jobs

Previous Next

Package: guix;

Reported by: Konrad Hinsen <konrad.hinsen <at> fastmail.net>

Date: Thu, 11 Aug 2022 08:41:02 UTC

Severity: normal

Done: Konrad Hinsen <konrad.hinsen <at> fastmail.net>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 57136 in the body.
You can then email your comments to 57136 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#57136; Package guix. (Thu, 11 Aug 2022 08:41:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konrad Hinsen <konrad.hinsen <at> fastmail.net>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Thu, 11 Aug 2022 08:41:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: bug-guix <at> gnu.org
Subject: Snakemake cannot execute remote jobs
Date: Thu, 11 Aug 2022 10:39:56 +0200
The execution of Snakemake workflows fails on a cluster because the
script that Snakemake executes remotely does not reference Python
correctly.

This is due to a patch applied in the Guix package
definition (build phase call-wrapper-not-wrapped-snakemake, 
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/python-xyz.scm#n9713)
which is outdated. The corresponding code in Snakemake was changed
significantly in the following commit:

  https://github.com/snakemake/snakemake/commit/e87cc979bea0567e1cd97722d385f472857df83c#diff-438f3317205fd7130727d0589d2fc1a6c2e1f6fc48c2c04d354a8a09b91ba2f4

Cheers,
  Konrad




Information forwarded to bug-guix <at> gnu.org:
bug#57136; Package guix. (Thu, 25 Aug 2022 15:36:02 GMT) Full text and rfc822 format available.

Message #8 received at 57136 <at> debbugs.gnu.org (full text, mbox):

From: Matthieu Simonin <matthieu.simonin <at> inria.fr>
To: 57136 <at> debbugs.gnu.org
Subject: Snakemake cannot execute remote jobs
Date: Thu, 25 Aug 2022 13:14:13 +0200
Hi there,

I've taken a stab at using it and restoring the patch as Konrad pointed seems the way to go.

But I also found that the package definition doesn't follow the guidelines[1] regarding the inputs definition.
One should use `propagated-inputs` instead of `inputs` for a python package.

In the current situation we can't invoke `snakemake` as a module:
`python -m snakemake` fails as the input dependencies aren't linked in the guix profile.

This way of invoking snakemake is used internally by snakemake here and there (for instance when calling the unit tests 
generated by `snakemake --generate-unit-tests`[2][3].

[1]: https://guix.gnu.org/manual/en/html_node/package-Reference.html
[2]: https://snakemake.readthedocs.io/en/stable/snakefiles/testing.html
[3]: https://github.com/snakemake/snakemake/blob/v7.7.0/snakemake/unit_tests/templates/ruletest.py.jinja2




Information forwarded to bug-guix <at> gnu.org:
bug#57136; Package guix. (Thu, 25 Aug 2022 18:27:02 GMT) Full text and rfc822 format available.

Message #11 received at 57136 <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: 57136 <at> debbugs.gnu.org
Subject: Re: Snakemake cannot execute remote jobs
Date: Thu, 25 Aug 2022 20:26:36 +0200
I have submitted a patch that fixes this problem:

   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57413

This is the patch that Matthieu referred to, and which he tested in a
cluster environment.

Cheers,
  Konrad




Information forwarded to bug-guix <at> gnu.org:
bug#57136; Package guix. (Thu, 25 Aug 2022 18:51:02 GMT) Full text and rfc822 format available.

Message #14 received at 57136 <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: 57136 <at> debbugs.gnu.org
Subject: Re: Snakemake cannot execute remote jobs
Date: Thu, 25 Aug 2022 20:50:12 +0200
The other problem that Matthieu pointed out (but which is
unrelated to the initial bug report) is fixed by the following
two patches for snakemake-6 and snakemake-7:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57414
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57415

Cheers,
  Konrad




Reply sent to Konrad Hinsen <konrad.hinsen <at> fastmail.net>:
You have taken responsibility. (Mon, 29 Aug 2022 16:51:02 GMT) Full text and rfc822 format available.

Notification sent to Konrad Hinsen <konrad.hinsen <at> fastmail.net>:
bug acknowledged by developer. (Mon, 29 Aug 2022 16:51:02 GMT) Full text and rfc822 format available.

Message #19 received at 57136-done <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: 57136-done <at> debbugs.gnu.org
Subject: Re: Snakemake cannot execute remote jobs
Date: Mon, 29 Aug 2022 18:50:41 +0200
The bug is fixed via the patch referenced above in commit 5831155175614726685edab7efa60ce48e4da1f5.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 27 Sep 2022 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 1 year and 183 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.