GNU logs - #31343, boring messages

Message sent to bug-guile@HIDDEN:

Subject: bug#31343: scm_c_primitive_load behavior/documentation bug
Resent-From: Tom Balzer <niebieskitrociny@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Wed, 02 May 2018 17:56:01 +0000
Resent-Message-ID: <handler.31343.B.152528373028185 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 31343 <at> debbugs.gnu.org
Cc: Tom Balzer <niebieskitrociny@HIDDEN>
User-agent: mu4e 1.0; emacs 25.3.1
From: Tom Balzer <niebieskitrociny@HIDDEN>
Message-ID: <877eom2c63.fsf@HIDDEN>
Date: Wed, 02 May 2018 12:35:54 -0500
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hello -

In ./guile/libguile/load.c, the function scm_c_primitive_load converts a
c string to a SCM string via scm_from_locale_string. I was reading the
manual and in section 6.6.5.14, it says:

> C Function: SCM scm_from_locale_string (const char *str)
> C Function: SCM scm_from_locale_stringn (const char *str, size_t
>
> [...]
>
> Note that these functions should _not_ be used to convert C string
> constants, because there is no guarantee that the current locale
> will match that of the execution character set, used for string and
> character constants.  Most modern C compilers use UTF-8 by default,
> so to convert C string constants we recommend
> ‘scm_from_utf8_string’.

This implies to me that you should not use scm_c_primitive_load with any
constant, like this:

#include <libguile.h>
#include <stdlib.h>

#define FILE "/home/niebie/sc/sdl/states.scm"

void *some_func(void *arg){
  SCM scm_c_primitive_load(FILE);

  return NULL;
}

int main(int argc, char **argv){
  void *res = scm_with_guile(some_func, NULL);
  return EXIT_SUCCESS;
}

I saw this only by reading the source for this function, as from the
documentation it isn't obvious. I am sending this to bug-guile because I
think that either this is a documentation bug or an implementation
bug. In either case I am happy to send a patch that fixes whichever is
at fault.

A counter example is scm_c_public_variable in modules.c, which uses
scm_from_utf8_symbol on the inputs, which precludes the use of dynamic c
strings for the input. Again, not something documented. I would think
both of these functions would do things the same way.

Thanks,
Tom

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Tom Balzer <niebieskitrociny@HIDDEN>
Subject: bug#31343: Acknowledgement (scm_c_primitive_load behavior/documentation
 bug)
Message-ID: <handler.31343.B.152528373028185.ack <at> debbugs.gnu.org>
References: <877eom2c63.fsf@HIDDEN>
X-Gnu-PR-Message: ack 31343
X-Gnu-PR-Package: guile
Reply-To: 31343 <at> debbugs.gnu.org
Date: Wed, 02 May 2018 17:56:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guile@HIDDEN

If you wish to submit further information on this problem, please
send it to 31343 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
31343: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D31343
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-guile@HIDDEN:

Subject: bug#31343: scm_c_primitive_load behavior/documentation bug
Resent-From: Mark H Weaver <mhw@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Mon, 28 May 2018 12:58:01 +0000
Resent-Message-ID: <handler.31343.B31343.152751223825803 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Tom Balzer <niebieskitrociny@HIDDEN>
Cc: 31343 <at> debbugs.gnu.org
From: Mark H Weaver <mhw@HIDDEN>
References: <877eom2c63.fsf@HIDDEN>
Date: Mon, 28 May 2018 08:55:56 -0400
In-Reply-To: <877eom2c63.fsf@HIDDEN> (Tom Balzer's message of "Wed, 02 May
 2018 12:35:54 -0500")
Message-ID: <87bmd0x7kz.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi Tom,

Sorry for taking so long to respond.

Tom Balzer <niebieskitrociny@HIDDEN> writes:

> In ./guile/libguile/load.c, the function scm_c_primitive_load converts a
> c string to a SCM string via scm_from_locale_string. I was reading the
> manual and in section 6.6.5.14, it says:
>
>> C Function: SCM scm_from_locale_string (const char *str)
>> C Function: SCM scm_from_locale_stringn (const char *str, size_t
>>
>> [...]
>>
>> Note that these functions should _not_ be used to convert C string
>> constants, because there is no guarantee that the current locale
>> will match that of the execution character set, used for string and
>> character constants.  Most modern C compilers use UTF-8 by default,
>> so to convert C string constants we recommend
>> =E2=80=98scm_from_utf8_string=E2=80=99.
>
> This implies to me that you should not use scm_c_primitive_load with any
> constant, like this:
>
> #include <libguile.h>
> #include <stdlib.h>
>
> #define FILE "/home/niebie/sc/sdl/states.scm"
>
> void *some_func(void *arg){
>   SCM scm_c_primitive_load(FILE);

If the C string literal contains only ASCII characters, then it doesn't
matter either way, because all C locale encodings are ASCII-compatible.
Perhaps we should make that more clear in the documentation that you
quoted above.

A related question is whether we should change the API of
'scm_c_primitive_load' to expect a UTF-8 encoded file name instead of a
locale encoded one.

If the file name comes from a C string literal, then it will probably be
UTF-8 encoded, because that's what modern compilers tend to do.  On the
other hand, if the file name comes from somewhere else, e.g. from user
input, POSIX command line arguments, or environment variables, then it
should probably be the locale encoding.

I'm inclined to leave 'scm_c_primitive_load' as it is, because the
expected encoding is effectively part of the API.  Some programs might
depend on its current behavior, and file names are reasonably likely to
come from sources like environment variables or command-line arguments.
Furthermore, file names in C string literals are quite likely to be
ASCII-only anyway.

> I saw this only by reading the source for this function, as from the
> documentation it isn't obvious. I am sending this to bug-guile because I
> think that either this is a documentation bug or an implementation
> bug. In either case I am happy to send a patch that fixes whichever is
> at fault.
>
> A counter example is scm_c_public_variable in modules.c, which uses
> scm_from_utf8_symbol on the inputs, which precludes the use of dynamic c
> strings for the input. Again, not something documented. I would think
> both of these functions would do things the same way.

I'm not sure, because file names are reasonably likely to come from
external sources that are likely to be locale encoded, whereas Scheme
variable names are overwhelmingly likely to be C string literals.

In any case, these are longstanding APIs, so I don't think we should
change them.

So, I think the proper fixes here are to the documentation.  As you
suggested, the documentation for 'scm_c_public_variable',
'scm_c_primitive_load', and all other C functions in our API should
specify the encoding for C string arguments.

If you'd like to work on it, I'd be glad to accept documentation fixes
along these lines.

     Thanks!
       Mark

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.