GNU bug report logs - #32544
[ELPA] core packages need generated files

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32544 in the body.
You can then email your comments to 32544 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: bug-gnu-emacs <at> gnu.org
Subject: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 17:05:10 +0200

An ELPA core package consist of files contained in the Emacs
repository. Sometimes, this is not sufficient: generated files like
"package-name".info or "dir" might also be required. It should be
possible to add them.


In GNU Emacs 27.0.50 (build 5, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-08-17 built on detlef
Repository revision: 9432b70f76e9e8c3f7c648cd9c93f56b16a0a1dd
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.1 LTS

Recent messages:
uncompressing 25446.gz...done
End of message [8 times]
Auto-saving...
Mark set [6 times]
Info-search: Search failed: "thread" [2 times]
Saving /home/albinus/.newsrc.eld...
Saving file /home/albinus/.newsrc.eld...
Wrote /home/albinus/.newsrc.eld
Saving /home/albinus/.newsrc.eld...done
Making completion list...

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8

Major mode: Group

Minor modes in effect:
  gnus-undo-mode: t
  diff-auto-refine-mode: t
  erc-notify-mode: t
  erc-notifications-mode: t
  display-time-mode: t
  shell-dirtrack-mode: t
  icomplete-mode: t
  show-paren-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/albinus/src/elpa/packages/debbugs/debbugs-org hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs-org
/home/albinus/src/elpa/packages/debbugs/debbugs-gnu hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs-gnu
/home/albinus/src/elpa/packages/debbugs/debbugs hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs
/home/albinus/src/elpa/packages/debbugs/debbugs-autoloads hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs-autoloads
/home/albinus/src/elpa/packages/debbugs/debbugs-pkg hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs-pkg
/home/albinus/src/elpa/packages/debbugs/debbugs-browse hides /home/albinus/.emacs.d/elpa/debbugs-0.15/debbugs-browse
~/lisp/telepathy hides /home/albinus/.emacs.d/elpa/telepathy-20131209.1258/telepathy
/home/albinus/src/elpa/packages/tramp-theme/tramp-theme hides /home/albinus/.emacs.d/elpa/tramp-theme-0.2/tramp-theme
/home/albinus/src/elpa/packages/tramp-theme/tramp-theme-autoloads hides /home/albinus/.emacs.d/elpa/tramp-theme-0.2/tramp-theme-autoloads
/home/albinus/src/elpa/packages/tramp-theme/tramp-theme-pkg hides /home/albinus/.emacs.d/elpa/tramp-theme-0.2/tramp-theme-pkg
~/src/tramp/lisp/tramp-smb hides /usr/local/src/emacs-test/lisp/net/tramp-smb
~/src/tramp/lisp/tramp-uu hides /usr/local/src/emacs-test/lisp/net/tramp-uu
~/src/tramp/lisp/tramp-adb hides /usr/local/src/emacs-test/lisp/net/tramp-adb
~/src/tramp/lisp/tramp-archive hides /usr/local/src/emacs-test/lisp/net/tramp-archive
~/src/tramp/lisp/tramp-cmds hides /usr/local/src/emacs-test/lisp/net/tramp-cmds
~/src/tramp/lisp/tramp-cache hides /usr/local/src/emacs-test/lisp/net/tramp-cache
~/src/tramp/lisp/trampver hides /usr/local/src/emacs-test/lisp/net/trampver
~/src/tramp/lisp/tramp-ftp hides /usr/local/src/emacs-test/lisp/net/tramp-ftp
~/src/tramp/lisp/tramp-sh hides /usr/local/src/emacs-test/lisp/net/tramp-sh
~/src/tramp/lisp/tramp hides /usr/local/src/emacs-test/lisp/net/tramp
~/src/tramp/lisp/tramp-loaddefs hides /usr/local/src/emacs-test/lisp/net/tramp-loaddefs
~/lisp/dbus hides /usr/local/src/emacs-test/lisp/net/dbus
~/src/tramp/lisp/tramp-gvfs hides /usr/local/src/emacs-test/lisp/net/tramp-gvfs
~/src/tramp/lisp/tramp-compat hides /usr/local/src/emacs-test/lisp/net/tramp-compat

Features:
(shadow emacsbug edebug archive-contents rect dired-aux ada-mode
which-func align find-file m4-mode grep make-mode sh-script smie
ediff-ptch lisp-mnt reporter tramp-cmds timezone files-x arc-mode
archive-mode hyperspec log-view eww mm-url nndoc debbugs-gnu debbugs
soap-client warnings rng-xsd rng-dt rng-util xsd-regexp debug backtrace
executable gnus-fun gnus-dup nnfolder nnmbox url-http url-gw url-auth
url-queue url-cache url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf org-table org-rmail org-mhe org-irc
org-info org-gnus org-docview doc-view image-mode org-bibtex bibtex
org-bbdb org-w3m org-element avl-tree generator org org-macro
org-footnote org-pcomplete org-list org-faces org-entities noutline
outline org-version ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob
ob-table ob-keys ob-exp ob-comint ob-core ob-eval org-compat org-macs
org-loaddefs cal-menu calendar cal-loaddefs mailalias nnir smerge-mode
whitespace tabify imenu man eieio-opt speedbar sb-image ezimage dframe
find-func cl-print help-fns radix-tree find-dired magit-utils crm
flow-fill sort gnus-cite smiley url-util shr-color color shr
text-property-search svg dom browse-url qp mm-archive mail-extr
gnus-async gnus-bcklg gnus-ml pop3 utf-7 nndraft nnmh nnml gnutls
network-stream nsm gnus-agent gnus-srvr gnus-score score-mode nnvirtual
gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig mailcap
gnus-cache gnus-sum nnnil smtpmail sendmail gnus-demon nntp gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader wid-edit
texinfo vc-hg em-xtra compile em-unix em-tramp em-term em-script
em-prompt em-ls em-hist em-pred em-glob em-dirs em-cmpl em-basic
em-banner em-alias esh-var esh-cmd esh-opt esh-io esh-ext esh-proc
esh-arg esh-groups eshell esh-module esh-mode esh-util time-stamp
misearch multi-isearch ediff-vers ediff-merg ediff-wind ediff-diff
ediff-mult ediff-help ediff-init ediff-util ediff autoconf autoconf-mode
bug-reference log-edit message rmc puny rfc822 mml mml-sec epa gnus-util
rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader pcvs-util add-log vc-git diff-mode easy-mmode
vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc-dir ewoc elec-pair
erc-notify erc-networks erc-desktop-notifications erc-match
notifications dbus xml erc-goodies erc erc-backend erc-compat thingatpt
pp erc-loaddefs cperl-mode time tramp-sh kubernetes-tramp lxc-tramp
docker-tramp tramp-cache vagrant-tramp dash term disp-table ehelp tramp
trampver tramp-compat tramp-loaddefs ucs-normalize shell pcomplete
comint ansi-color ring parse-time format-spec advice ido jka-compr
icomplete paren vc vc-dispatcher dired dired-loaddefs time-date
mule-util rx edmacro kmacro slime-autoloads info finder-inf package
let-alist derived pcase cl-extra help-mode easymenu url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq byte-opt gv bytecomp byte-compile
cconv epg epg-config subr-x cl-loaddefs cl-lib tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 2759940 445277)
 (symbols 48 69831 7)
 (strings 32 351088 101295)
 (string-bytes 1 27282365)
 (vectors 16 99697)
 (vector-slots 8 2318382 193878)
 (floats 8 3288 1453)
 (intervals 56 312043 6513)
 (buffers 992 262))

Message #8 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 32544 <at> debbugs.gnu.org
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 11:21:16 -0400

> An ELPA core package consist of files contained in the Emacs
> repository.  Sometimes, this is not sufficient: generated files like
> "package-name".info or "dir" might also be required.  It should be
> possible to add them.

Note that this same need exists for non-core packages.
The main issue is to make it possible to build those files on
elpa.gnu.org without too many security risks.

I planned to do that by running "make" inside an LXC container, but my
attempts to make a lightweight LXC container on elpa.gnu.org failed
se far.
Any help with this sysadmin side of things would be very welcome.


        Stefan

Message #13 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: Philippe Vaucher <philippe.vaucher <at> gmail.com>, 32544 <at> debbugs.gnu.org
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 17:34:28 +0200

Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:

> I planned to do that by running "make" inside an LXC container, but my
> attempts to make a lightweight LXC container on elpa.gnu.org failed se
> far.

For core packages, there would be the problem then where to get the
Makefile from.

> Any help with this sysadmin side of things would be very welcome.

Unfortunately, I also haven't enough experience with containers in order
to be able to help you. There is the Emacs Docker project
<https://hub.docker.com/r/silex/emacs>. Maybe the author, Philippe
Vaucher, could help (I've added him in Cc).

>         Stefan

Best regards, Michael.

Message #16 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Philippe Vaucher <philippe.vaucher <at> gmail.com>, 32544 <at> debbugs.gnu.org
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 11:52:24 -0400

>> I planned to do that by running "make" inside an LXC container, but my
>> attempts to make a lightweight LXC container on elpa.gnu.org failed se
>> far.
> For core packages, there would be the problem then where to get the
> Makefile from.

Yes, for core packages, there are additional issues, but I think they'd
be easy enough to address (we just need to choose where to keep that
extra Makefile).

>> Any help with this sysadmin side of things would be very welcome.
> Unfortunately, I also haven't enough experience with containers in order
> to be able to help you. There is the Emacs Docker project
> <https://hub.docker.com/r/silex/emacs>. Maybe the author, Philippe
> Vaucher, could help (I've added him in Cc).

Note that we don't need to run Emacs: we need to run makeinfo (and
potentially many other tools).  I tried to make an LXC container that
simply re-uses the host's filesystem (so it ends up barely more costly
than a chroot while providing a complete Debian system) and managed to
get it running on my own machine, but not on elpa.gnu.org.


        Stefan

Message #19 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 32544 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 19:31:07 -0400

Stefan Monnier wrote:

> The main issue is to make it possible to build those files on
> elpa.gnu.org without too many security risks.
>
> I planned to do that by running "make" inside an LXC container, but my
> attempts to make a lightweight LXC container on elpa.gnu.org failed
> se far.

Is the concern privilege escalation in build recipes in malicious elpa
packages?

But couldn't the same package run the same bad code at package install
time on the end user's machine, today and for as long as elpa.gnu.org
has existed?

Ie, if we assume malicious code can get into elpa packages with no-one
noticing, the whole system is already broken anyway?

So would it be good enough in practice to avoid accidental damage by
running make as a dedicated elpa-build user with no special privs?

But if you want to make the elpa system more secure one piece at a time,
that's obviously no bad thing.

Message #22 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 32544 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 20:15:28 -0400

> Is the concern privilege escalation in build recipes in malicious elpa
> packages?

Yes.

> But couldn't the same package run the same bad code at package install
> time on the end user's machine, today and for as long as elpa.gnu.org
> has existed?

Yes.  Tho not only "at package install time", since that same bad code
can be run any time later when the package is activated or loaded...

> Ie, if we assume malicious code can get into elpa packages with no-one
> noticing, the whole system is already broken anyway?

Yup.

> But if you want to make the elpa system more secure one piece at a time,
> that's obviously no bad thing.

I think the reasons why I'm more worried about elpa.gnu.org than the
end-user's machines include:

- very little time between the moment we receive the commit-diffs by
  email and the moment the code is run.  So even if we notice the
  offending code on the spot, there's not much time to react.
- elpa.gnu.org is part of infrastructure that Emacs users trust when
  downloading GNU ELPA packages (e.g. it holds the PGP signing key), so
  a breach could affect all GNU ELPA users (especially if not
  noticed).


        Stefan

Message #25 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 32544 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Mon, 27 Aug 2018 22:13:06 -0400

Stefan Monnier wrote:

> I think the reasons why I'm more worried about elpa.gnu.org than the
> end-user's machines include:
>
> - very little time between the moment we receive the commit-diffs by
>   email and the moment the code is run.  So even if we notice the
>   offending code on the spot, there's not much time to react.
> - elpa.gnu.org is part of infrastructure that Emacs users trust when
>   downloading GNU ELPA packages (e.g. it holds the PGP signing key), so
>   a breach could affect all GNU ELPA users (especially if not
>   noticed).

Sounds very sensible, best of luck! :)

Message #28 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 32544 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Tue, 28 Aug 2018 07:54:46 -0400

>> I think the reasons why I'm more worried about elpa.gnu.org than the
>> end-user's machines include:
>>
>> - very little time between the moment we receive the commit-diffs by
>>   email and the moment the code is run.  So even if we notice the
>>   offending code on the spot, there's not much time to react.
>> - elpa.gnu.org is part of infrastructure that Emacs users trust when
>>   downloading GNU ELPA packages (e.g. it holds the PGP signing key), so
>>   a breach could affect all GNU ELPA users (especially if not
>>   noticed).

One more reason:

- elpa.gnu.org *can* run that code in a sandbox, whereas the end-user
  really wants to run the package's code in his "real" system (or
  otherwise would need to run his whole Emacs session in a sandbox).

> Sounds very sensible, best of luck! :)

Hmm... looks like you forgot to attach the patch to your message.
Could you send it again, please?


        Stefan

Message #31 received at 32544 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 32544 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Tue, 28 Aug 2018 17:17:00 +0200

[Message part 1 (text/plain, inline)]

On Mon, Aug 27, 2018, 17:52 Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:

>
> >> Any help with this sysadmin side of things would be very welcome.
> > Unfortunately, I also haven't enough experience with containers in order
> > to be able to help you. There is the Emacs Docker project
> > <https://hub.docker.com/r/silex/emacs>. Maybe the author, Philippe
> > Vaucher, could help (I've added him in Cc).
>
> Note that we don't need to run Emacs: we need to run makeinfo (and
> potentially many other tools).  I tried to make an LXC container that
> simply re-uses the host's filesystem (so it ends up barely more costly
> than a chroot while providing a complete Debian system) and managed to
> get it running on my own machine, but not on elpa.gnu.org.


Hello! I'd love to help but I don't understand what the issue is. For
information Emacs does not build in docker out of the box because it needs
to disable ASLR (see https://github.com/moby/moby/issues/22801).

Tell me what you need or what precise problem you are encountering and
maybe I can clarify what needs to be done.

Kind regards,
Philippe

[Message part 2 (text/html, inline)]

Message #36 received at 32544-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 32544-done <at> debbugs.gnu.org
Subject: Re: bug#32544: [ELPA] core packages need generated files
Date: Sun, 20 Dec 2020 12:20:18 -0500

>> An ELPA core package consist of files contained in the Emacs
>> repository.  Sometimes, this is not sufficient: generated files like
>> "package-name".info or "dir" might also be required.  It should be
>> possible to add them.
> Note that this same need exists for non-core packages.

This is now supported in elpa.gnu.org (via `:doc` and `:make`
properties in the `elpa-packages` specification).


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.