GNU bug report logs - #46231
Add emilua

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46231 in the body.
You can then email your comments to 46231 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
To: guix-patches <at> gnu.org
Subject: Add emilua
Date: Mon, 1 Feb 2021 07:00:42 -0300

Hi,

I've created a new package for guix. Here's its definition:
https://gitlab.com/emilua/emilua/-/snippets/2068658

Please let me know if it has any issues so I can fix them until it's
ready for inclusion in the main repository.

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Message #8 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
To: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Sun, 21 Feb 2021 02:21:42 +0100

Hello,

Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com> writes:

> I've created a new package for guix. Here's its definition:
> https://gitlab.com/emilua/emilua/-/snippets/2068658

Thank you!

> Please let me know if it has any issues so I can fix them until it's
> ready for inclusion in the main repository.

There are some issue to fix:
- the name of the custom luajit2 package should be changed. Maybe
  openresty-luajit?
- that package should have a proper version and build tag out of it.
- I see that emilua uses #:recursive? set to #true. Could some
  submodules could be unbundled by any chance?
- You disabled tests. I would be nice to provide a reason for that.
- You don't need `string-append' in the description. Just write the full
  textin a single string.

Could you split this into two patches and send them here, for easier review?

Regards,
-- 
Nicolas Goaziou

Message #11 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
To: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Sun, 21 Feb 2021 09:04:25 -0300

Em sáb., 20 de fev. de 2021 às 22:21, Nicolas Goaziou
<mail <at> nicolasgoaziou.fr> escreveu:
> There are some issue to fix:
> - the name of the custom luajit2 package should be changed. Maybe
>   openresty-luajit?
> - that package should have a proper version and build tag out of it.

I've used "define" instead "define-public" so it wouldn't be really
available elsewhere in the system. However after some thought I do
agree that this dependency can be generally useful to more people and
should have a proper name.

What do you think about the name openresty-luajit-lua52?

> - I see that emilua uses #:recursive? set to #true. Could some
>   submodules could be unbundled by any chance?

Some, yes, but not all. A Debian packager asked me the same and I
already started to unbundle the dependencies where it can make sense
to use the one found in the system. I can offer more details on why a
few dependencies still must be provided as submodules if requested.
Next emilua version (a couple of weeks away) should use more
dependencies from the system and less bundled dependencies, but you'll
still need #:recursive? set to #true for the bundled dependencies that
will remain.

> - You disabled tests. I would be nice to provide a reason for that.

This is how the tests are run:

1. The emilua binary is generated.
2. The emilua binary is executed against every *.lua file inside the test dir.
3. An output is generated and compared against the matching *.out file
inside the same dir.
4. If they differ, the test fails.

There are two types of tests:

* Good test cases. No error expected.
* Bad test cases. Buggy code is feeded to the runtime to see if it is
able to detect the error.

On exceptions (the bad test cases), Lua throws an error string and
Emilua will print the stacktrace. The stacktrace is generated by the
Lua VM and contains paths to the files involved. This path will be
different in every machine you run the tests as you probably will
store the source code in different paths. Here's an example:

$ emilua test/interrupt7.lua
Main fiber from VM 0x7f1f133d1380 panicked: 'Invalid argument'
stack traceback:
  /home/me/emilua/test/interrupt7.lua:5: in main chunk
  [C]: in function ''
  [string "?"]: in function <[string "?"]:0>

So I filter the test output using an AWK script that replaces the
filename by "input.lua". Here's the translation line:
https://gitlab.com/emilua/emilua/-/blob/ded59fa9d4bb8977aed40fb7b683c6403b462f28/test/run_test.awk#L60

This filter ensures that a path in the likes of
"/home/me/emilua/test/interrupt7.lua" will be spit as "input.lua".
That makes the output predictable so I can write the *.out files that
reference input.lua. Here's the matching test/interrupt7.out from the
previous run:

Main fiber from VM 0x1 panicked: 'Invalid argument'
stack traceback:
  input.lua:5: in main chunk
  [C]: in function ''
  [string "?"]: in function <[string "?"]:0>

Unfortunately this strategy is not enough. The Lua VM will shorten
paths by excluding some of its parts and replace them by "..." when
they are too long. That's why the script has this rule:
https://gitlab.com/emilua/emilua/-/blob/ded59fa9d4bb8977aed40fb7b683c6403b462f28/test/run_test.awk#L4

However running the tests from the guix build unveiled yet another
problem with this script. If even longer paths are at play (as in the
guix build), the Lua VM will apply a different shortening algorithm to
the paths sometimes. So the tests run, the emilua binary will do the
right thing, but the translation script won't work on one of the test
cases and will raise a false negative. I intend to solve this issue
and I already started to investigate so it'll eventually be solved,
but in the meantime I just disabled the tests. I don't control the
paths the Lua VM generates so improving the AWK script is really my
only choice here.

> - You don't need `string-append' in the description. Just write the full
>   textin a single string.

Okay.

> Could you split this into two patches and send them here, for easier review?

Sure. Let's get the openresty-luajit-lua52 package in first and then
I'll work on the second patch.

Here's the patch for openresty-luajit-lua52 alone:
https://gitlab.com/emilua/emilua/-/snippets/2079960


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Message #14 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
To: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Mon, 22 Feb 2021 11:04:42 +0100

Hello,

Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com> writes:

Thank you for your detailed answer! Some comments follow.

> I've used "define" instead "define-public" so it wouldn't be really
> available elsewhere in the system.

I know. But this does not prevent to choose an explicit name.

> However after some thought I do agree that this dependency can be
> generally useful to more people and should have a proper name.
>
> What do you think about the name openresty-luajit-lua52?

The trend in other distributions is to use luajit-openresty, in this
order. So maybe luajit-lua52-openresty? Or luajit-openresty if that is
unambiguous. It is not a big issue anyway.

>> - I see that emilua uses #:recursive? set to #true. Could some
>>   submodules could be unbundled by any chance?
>
> Some, yes, but not all. A Debian packager asked me the same and I
> already started to unbundle the dependencies where it can make sense
> to use the one found in the system. I can offer more details on why a
> few dependencies still must be provided as submodules if requested.
> Next emilua version (a couple of weeks away) should use more
> dependencies from the system and less bundled dependencies, but you'll
> still need #:recursive? set to #true for the bundled dependencies that
> will remain.

Then could you add a comment explaining what will be required, and what
can be removed later on (as a TODO)?

> I intend to solve this issue and I already started to investigate so
> it'll eventually be solved, but in the meantime I just disabled the
> tests. I don't control the paths the Lua VM generates so improving the
> AWK script is really my only choice here.

OK. Then could you add a short comment explaining the issue, and that it
is being worked on?

> Sure. Let's get the openresty-luajit-lua52 package in first and then
> I'll work on the second patch.

Great!

> Here's the patch for openresty-luajit-lua52 alone:
> https://gitlab.com/emilua/emilua/-/snippets/2079960

The version should be "2.1-20201027" and the commit (string-append "v"
version). Don't forget to remove the string-append in the description.
Also, the description itself is not particularly clear. Maybe

     This package provides the official OpenResty branch of LuaJIT.  It
     is not to be considered a fork, since changes are regularly
     synchronized from the upstream LuaJIT project.

Could you send this to this ML using git format-patch?

Regards,
-- 
Nicolas Goaziou

Message #17 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
To: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Tue, 23 Feb 2021 17:35:07 -0300

[Message part 1 (text/plain, inline)]

Em seg., 22 de fev. de 2021 às 07:04, Nicolas Goaziou
<mail <at> nicolasgoaziou.fr> escreveu:
> The trend in other distributions is to use luajit-openresty, in this
> order. So maybe luajit-lua52-openresty? Or luajit-openresty if that is
> unambiguous. It is not a big issue anyway.

luajit-openresty might be ambiguous as not every package might depend
on the lua52 configure flag, so I renamed it to
luajit-lua52-openresty.

> Could you send this to this ML using git format-patch?

You may find the patch with the requested fixes attached to this
email. I'll implement the changes to the other package once we have
luajit-lua52-openresty in place.

Thanks for acting as the reviewer of this patch.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

[0001-Add-luajit-lua52-openresty.patch (text/x-patch, attachment)]

Message #20 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
To: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Wed, 24 Feb 2021 14:07:33 +0100

Hello,

Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com> writes:

> luajit-openresty might be ambiguous as not every package might depend
> on the lua52 configure flag, so I renamed it to
> luajit-lua52-openresty.

OK.

> You may find the patch with the requested fixes attached to this
> email. I'll implement the changes to the other package once we have
> luajit-lua52-openresty in place.

You didn't include "luajit-enable-lua52compat.patch". Could you send it
too?

Also, there is a new release: 2.1-20201229. You may want to send an
updated version. I can also update it.

Thanks!

Regards,
-- 
Nicolas Goaziou

Message #23 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
To: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Wed, 24 Feb 2021 10:20:19 -0300

[Message part 1 (text/plain, inline)]

Em qua., 24 de fev. de 2021 às 10:07, Nicolas Goaziou
<mail <at> nicolasgoaziou.fr> escreveu:
> > You may find the patch with the requested fixes attached to this
> > email. I'll implement the changes to the other package once we have
> > luajit-lua52-openresty in place.
>
> You didn't include "luajit-enable-lua52compat.patch". Could you send it
> too?

Oops, my bad. You may find it attached to this email.

> Also, there is a new release: 2.1-20201229. You may want to send an
> updated version. I can also update it.

I'll let you do the update then.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

[luajit-enable-lua52compat.patch (text/x-patch, attachment)]

Message #26 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
To: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Wed, 24 Feb 2021 16:49:41 +0100

Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com> writes:

> Oops, my bad. You may find it attached to this email.

OK. Since it was trivial, I used a phase instead.

>> Also, there is a new release: 2.1-20201229. You may want to send an
>> updated version. I can also update it.
>
> I'll let you do the update then.

Done.

I applied the patch. We can now proceed with emilua.

Thanks!

Regards,

Message #29 received at 46231 <at> debbugs.gnu.org (full text, mbox):

From: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
To: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
Cc: 46231 <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Thu, 25 Feb 2021 11:11:22 -0300

[Message part 1 (text/plain, inline)]

Em qua., 24 de fev. de 2021 às 12:49, Nicolas Goaziou
<mail <at> nicolasgoaziou.fr> escreveu:
> > I'll let you do the update then.
> Done.

Thanks.

> I applied the patch. We can now proceed with emilua.

You may find the updated patch (using git format) attached in this email.

Btw, if you don't mind, I'd like to use the space to also make a few
questions guix-related. Emilua is a software similar to NodeJS, but it
doesn't have its own npm, pip, gem, or anything alike. I'd like to use
guix as the package manager in emilua. I don't want to create another
npm. Therefore all the code I'm writing for the emilua's next version
was done with guix in mind.

I've learned about native-search-paths and how it can be used to fill
environment variables such as GST_PLUGIN_PATH. I've learned about the
approach to "merge" directories from each package to fill the guix
environment/profile and how crucial it is for such environment
variables to guide the software running inside it (e.g. python and
PYTHONPATH). That's pretty much solved for me (unless there are other
approaches in guix that I'm missing and they would fit better).

Next step for me is to better understand what's the proper take when
dealing with GStreamer native-search-paths and pkg-config. I'm talking
about GStreamer here because it faces a similar problem and I believe
it's easier to talk on issues from existing and packaged software.

GStreamer defines native-search-paths to
GST_PLUGIN_SYSTEM_PATH=lib/gstreamer-1.0. As far as I tested, if a
package gstreamer-foobar depends on gstreamer and install files to
${prefix}/lib/gstreamer-1.0, then GST_PLUGIN_SYSTEM_PATH will be
automatically updated to also contain gstreamer-foobar's
/gnu/store/*/lib/gstreamer-1.0.

My question here is: how should the software packaged in
gstreamer-foobar find out the proper directory to install its plugins?
Should it be ${prefix}/lib/gstreamer-1.0? Should it be
${prefix}/lib/gstreamer-1.1? That's something that shouldn't be
hardcoded. As far as I see, it can just use the pluginsdir from
GStreamer's pkg-config definition. Like so:

$ pkg-config --variable=pluginsdir gstreamer-1.0

However this command will print the wrong dir in guix. It'll print
something like:

/gnu/store/9if71w58d5mkxfxyc7fpz289qssnkqsv-gstreamer-1.18.2/lib/gstreamer-1.0

But that's the "namespace" for the gstreamer package, not for the
gstreamer-foobar package. A solution would be to invoke pkg-config as
follows:

$ pkg-config --define-variable="prefix=${prefix}"
--variable=pluginsdir gstreamer-1.0

This will actually print the proper ${prefix}/lib/gstreamer-1.0. But
that's my question here. How should pkg-config be invoked properly? I
could just as well invoke it as:

$ pkg-config --define-variable="prefix=${prefix}"
--define-variable=libdir='${prefix}/lib' --variable=pluginsdir
gstreamer-1.0


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

[0001-Add-emilua.patch (text/x-patch, attachment)]

Message #34 received at 46231-done <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
To: Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com>
Cc: 46231-done <at> debbugs.gnu.org
Subject: Re: [bug#46231] Add emilua
Date: Sat, 27 Feb 2021 12:20:28 +0100

Hello,

Vinícius dos Santos Oliveira <vini.ipsmaker <at> gmail.com> writes:

> You may find the updated patch (using git format) attached in this
> email.

Thank you!

I re-ordered inputs alphabetically, fixed comment syntax (Lisp uses two
semicolons for full line comments), added a copyright line and a proper
commit message, and applied your patch.

> Btw, if you don't mind, I'd like to use the space to also make a few
> questions guix-related.

I think a more appropriate place would be Guix Devel mailing list
<mailto:guix-devel <at> gnu.org>. May I ask you to post again your questions
there? Your message will get more visibility from people way more
competent than I am on this subject.

Regards,
-- 
Nicolas Goaziou

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.