GNU bug report logs - #14599
An option to make vector allocation aligned

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Package: guile; Severity: wishlist; Reported by: Jan Schukat <shookie@HIDDEN>; dated Wed, 12 Jun 2013 13:38:02 UTC; Maintainer for guile is bug-guile@HIDDEN.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 17 Jun 2013 10:04:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 17 06:04:17 2013
Received: from localhost ([127.0.0.1]:49873 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UoWIL-0004nv-At
	for submit <at> debbugs.gnu.org; Mon, 17 Jun 2013 06:04:17 -0400
Received: from mout.web.de ([212.227.17.11]:52478)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>) id 1UoWII-0004nc-20
 for 14599 <at> debbugs.gnu.org; Mon, 17 Jun 2013 06:04:15 -0400
Received: from [192.168.0.27] ([85.177.95.148]) by smtp.web.de (mrweb102) with
 ESMTPSA (Nemesis) id 0M5OYl-1UOhwH3YEB-00zXjr;
 Mon, 17 Jun 2013 12:04:07 +0200
Message-ID: <51BEDF15.4050302@HIDDEN>
Date: Mon, 17 Jun 2013 12:04:05 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: =?UTF-8?B?THVkb3ZpYyBDb3VydMOocw==?= <ludo@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
 <51B8E4B7.9060306@HIDDEN> <874nd2gmsq.fsf@HIDDEN>
 <CAN3veRddPmNy=JJ-J85cbhnycXD2X5h8ftjmJu5J9fKTxAfVTg@HIDDEN>
 <51BAD514.4090002@HIDDEN> <8761xgevcd.fsf@HIDDEN>
In-Reply-To: <8761xgevcd.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:R8QZGjrHPZ5vxEW5U2itUgta0u63x51zuiDl1PsJyaczCiWQQ5Z
 FO+zoQ/cY0RijRNItiBwINtu0Y5volVGNjoR8ekAR5dD3SUSSSMwKSn0bh74MACTr8G1tdr
 2Hx0bsZdc1OoXm6/nqVTHyAKeorMsATa07WedWKtsOMdTKoVm+cOrqNqrSYkTbYBEpcfqWD
 LiVyp41HPR+5LRSmI3Z0g==
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org, Daniel Hartwig <mandyke@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

On 06/14/2013 02:21 PM, Ludovic Courtès wrote:
> Jan Schukat <shookie@HIDDEN> skribis:
>
>>   The more I think about it and hear what you have to say, the more I
>> think alignment just needs to be tied to the type of the uniform
>> array.
> I think it would be wrong.  An array of floats is an array of floats,
> regardless of its alignment.

For floats nothing would change anyway. Those are 4byte data types with 
4byte alignment already. As for the larger types, you can still make 
them in any alignment with the take functions. Just the default would 
the optimized one when you add padding, and that's what it should be, 
since the scheme programmer doesn't care about the underlying memory 
layouts as much as the C programmer does. In normal guile uniform vector 
use you are completely oblivious to the underlying vector 
implementation, apart from the performance. In short: an array of floats 
is still an array of floats, and you can create them at any alignment, 
although cumbersomely. But the very point if creating arrays of native 
types is to better use the underlying hardware. And not taking advantage 
of alignment there unless you absolutely can't is negligent at best.
>> Also, now I lean more towards switching to 2.2 for myself and
>> implement it on there, because as Ludovic said, the compiling will
>> possibly preserve alignment there better.
> Well yeah, though you’d still need to come up with an annotation for
> that, and I’m not enthusiastic about changing the read syntax for that
> purpose.

You don't need a special annotation if the default alignment is the 
native alignment of the native type. If you create arrays of native 
types that is what you want in the vast majority of cases. Overriding 
that should be the extra work and forced by external requirements, not 
by internal coincidental implementation details. And the interfaces to 
do that are already there.

I wanna use native SIMD types, which are obviously less portable, but in 
the end where they do exist they are all more or less the same in memory 
layout:  4 32bit ieee floats, 4 32bit ints or 2 ieee doubles, all 
preferably aligned at 128 bit. That's true for sse, altivec and neon on 
x86, power and arm respectively.

I can see why you wouldn't want SIMD types in the core modules. I can 
also see why you wouldn't want to change existing read syntax.

What I can't see is why you wouldn't want native type arrays in native 
alignment by default. There is no downside, and the implementation makes 
doing that trivial.

And having a module of uniform SIMD type arrays/vectors could be very 
valuable. I'll probably do that as an extension then, with a #, reader. 
Would just be a lot of code duplication and less nice.


>
> Now, I think the compiler should support a generic annotation mechanism,
> to allow users to specify various things (like ‘declare’ in some
> implementations.)  That could be one possible use.
>
> Ludo’.

Yes, that would have a lot of utility.

Regards

Jan Schukat
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 14 Jun 2013 12:21:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 14 08:21:46 2013
Received: from localhost ([127.0.0.1]:43456 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UnT0j-0008Eo-1p
	for submit <at> debbugs.gnu.org; Fri, 14 Jun 2013 08:21:45 -0400
Received: from hera.aquilenet.fr ([141.255.128.1]:50301)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@HIDDEN>) id 1UnT0f-0008ES-96
 for 14599 <at> debbugs.gnu.org; Fri, 14 Jun 2013 08:21:42 -0400
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id A83C6D61;
 Fri, 14 Jun 2013 14:21:39 +0200 (CEST)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id BMaA2EX8vDVn; Fri, 14 Jun 2013 14:21:39 +0200 (CEST)
Received: from pluto (reverse-83.fdn.fr [80.67.176.83])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id 2462E9B3;
 Fri, 14 Jun 2013 14:21:39 +0200 (CEST)
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Jan Schukat <shookie@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
 <51B8E4B7.9060306@HIDDEN> <874nd2gmsq.fsf@HIDDEN>
 <CAN3veRddPmNy=JJ-J85cbhnycXD2X5h8ftjmJu5J9fKTxAfVTg@HIDDEN>
 <51BAD514.4090002@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 26 Prairial an 221 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0xEA52ECF4
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA  D15D 77DD 95E2 EA52 ECF4
X-OS: x86_64-unknown-linux-gnu
Date: Fri, 14 Jun 2013 14:21:38 +0200
In-Reply-To: <51BAD514.4090002@HIDDEN> (Jan Schukat's message of "Fri, 14
 Jun 2013 10:32:20 +0200")
Message-ID: <8761xgevcd.fsf@HIDDEN>
User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org, Daniel Hartwig <mandyke@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

Jan Schukat <shookie@HIDDEN> skribis:

>  The more I think about it and hear what you have to say, the more I
> think alignment just needs to be tied to the type of the uniform
> array.

I think it would be wrong.  An array of floats is an array of floats,
regardless of its alignment.

> Also, now I lean more towards switching to 2.2 for myself and
> implement it on there, because as Ludovic said, the compiling will
> possibly preserve alignment there better.

Well yeah, though you=E2=80=99d still need to come up with an annotation for
that, and I=E2=80=99m not enthusiastic about changing the read syntax for t=
hat
purpose.

Now, I think the compiler should support a generic annotation mechanism,
to allow users to specify various things (like =E2=80=98declare=E2=80=99 in=
 some
implementations.)  That could be one possible use.

Ludo=E2=80=99.
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 14 Jun 2013 08:32:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 14 04:32:33 2013
Received: from localhost ([127.0.0.1]:43145 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UnPQs-0004gC-Vt
	for submit <at> debbugs.gnu.org; Fri, 14 Jun 2013 04:32:32 -0400
Received: from mout.web.de ([212.227.17.11]:61927)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>) id 1UnPQq-0004fv-6B
 for 14599 <at> debbugs.gnu.org; Fri, 14 Jun 2013 04:32:29 -0400
Received: from [192.168.0.27] ([85.177.94.175]) by smtp.web.de (mrweb101) with
 ESMTPSA (Nemesis) id 0MFtH8-1Uam7H1ZFG-00Etl4;
 Fri, 14 Jun 2013 10:32:21 +0200
Message-ID: <51BAD514.4090002@HIDDEN>
Date: Fri, 14 Jun 2013 10:32:20 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: Daniel Hartwig <mandyke@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
 <51B8E4B7.9060306@HIDDEN> <874nd2gmsq.fsf@HIDDEN>
 <CAN3veRddPmNy=JJ-J85cbhnycXD2X5h8ftjmJu5J9fKTxAfVTg@HIDDEN>
In-Reply-To: <CAN3veRddPmNy=JJ-J85cbhnycXD2X5h8ftjmJu5J9fKTxAfVTg@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:yGKYSp8jHXtVM9h/PV7e6W1sc5dXHGQ8yV56fbERbr+0rjwJsJm
 tXFGcIcIUX2/x4ZTAMkYmYkYxx7NpTAG9gHOjmyH6zVAq0HifI0GtCEM7riBiKrZ4t4J9K7
 L6Y9ZDmeqYVF14IyCOX/tQD9Uc8dPX82GzreDImWi/MJLNUXT9xgNyxhD/x5WCJcRN8cxAd
 GxDNOPLAvm/817ycchFYQ==
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 14599
Cc: =?UTF-8?B?THVkb3ZpYyBDb3VydMOocw==?= <ludo@HIDDEN>, 14599 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

On 06/14/2013 03:33 AM, Daniel Hartwig wrote:
> On 13 June 2013 21:31, Ludovic Courtès <ludo@HIDDEN> wrote:
>> Jan Schukat <shookie@HIDDEN> skribis:
>>> The other question is the read syntax (one of the primary reasons I'm
>>> doing all this). If alignment is something that should be preserved in
>>> the permanent representation, you also need to store it in the flags,
>>> since the content pointer can be aligned by coincidence. I haven't
>>> looked at the compiling of bytevectors yet, to see if alignment can be
>>> handled easily there.
>> I agree that we’d need some sort of annotation to specify the alignment
>> of literals, but adding read syntax for that scares me somewhat.  What
>> do people think?
> I agree.  The read syntax for vector-ish types in guile is already
> large enough.  If alignment is important then use a procedural
> constructor and query.
>
> Alignment information not need to be printed with the default
> representation (read syntax), we dont also print the storage address,
> etc..
>
> Regards
  The more I think about it and hear what you have to say, the more I 
think alignment just needs to be tied to the type of the uniform array. 
Up to float and int 32 arrays nothing will change then. Double and int64 
arrays get one word of padding on 32 bit machines to make them 8 byte 
aligned. And then introduce new type flags m128 and m256 for for simd 
types that are 16 or 32 byte bit aligned, possibly the complex arrays 
too. Since you can interpret uniform arrays as all types of uniform 
array this should solve all alignment problems where needed. The simd 
type arrays must be able to accept and recognize int and float 
immediates though, and you must be able to group them. That's not really 
much new syntax, and won't interfere with the old syntax.

Also, now I lean more towards switching to 2.2 for myself and implement 
it on there, because as Ludovic said, the compiling will possibly 
preserve alignment there better.

Regards

Jan Schukat
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 14 Jun 2013 01:33:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 13 21:33:46 2013
Received: from localhost ([127.0.0.1]:42643 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UnItd-0007Nm-Ke
	for submit <at> debbugs.gnu.org; Thu, 13 Jun 2013 21:33:46 -0400
Received: from mail-ie0-f172.google.com ([209.85.223.172]:39042)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <mandyke@HIDDEN>) id 1UnItb-0007NY-4H
 for 14599 <at> debbugs.gnu.org; Thu, 13 Jun 2013 21:33:43 -0400
Received: by mail-ie0-f172.google.com with SMTP id 16so93872iea.17
 for <14599 <at> debbugs.gnu.org>; Thu, 13 Jun 2013 18:33:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type:content-transfer-encoding;
 bh=Wf5t7ONr64xlJ4cEbg+Lk5t4SOfGYQ9fYRyhApL38CE=;
 b=AZCpxocd64sG7U/2iqmuvxxEATDokhp5G0bLmPKmWRjWKCjXURvLp2gwaD2SrRphN/
 7N6KP0RtirjmvJ0QF6fJ7EEYH1hpfiVyfZpR2tu7QALrGhI8/ZWpti70lSSVLr3GA558
 T1HwWZLiD6FtsbZdhSPVmo/mVyXsmTfxOMG6q1URHxheWmoFwo0qFKOGhseW/42tSDDT
 /ewML+y6Ut42GlVdhXWxBOwka2i+An7a7bEhktcXp9H5P/LEK+KdhRAtga98OtTydVj2
 nfMvHC7oAjxJERt/E/V1jG+cTNZqKlr0QSQPqwIdo+RGbAy6Z3oGtfWdO+ZkM1YNmqHi
 9sYA==
MIME-Version: 1.0
X-Received: by 10.50.132.98 with SMTP id ot2mr28173igb.38.1371173617199; Thu,
 13 Jun 2013 18:33:37 -0700 (PDT)
Received: by 10.64.238.6 with HTTP; Thu, 13 Jun 2013 18:33:37 -0700 (PDT)
In-Reply-To: <874nd2gmsq.fsf@HIDDEN>
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
 <51B8E4B7.9060306@HIDDEN> <874nd2gmsq.fsf@HIDDEN>
Date: Fri, 14 Jun 2013 09:33:37 +0800
Message-ID: <CAN3veRddPmNy=JJ-J85cbhnycXD2X5h8ftjmJu5J9fKTxAfVTg@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
From: Daniel Hartwig <mandyke@HIDDEN>
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org, Jan Schukat <shookie@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

On 13 June 2013 21:31, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
> Jan Schukat <shookie@HIDDEN> skribis:
>> The other question is the read syntax (one of the primary reasons I'm
>> doing all this). If alignment is something that should be preserved in
>> the permanent representation, you also need to store it in the flags,
>> since the content pointer can be aligned by coincidence. I haven't
>> looked at the compiling of bytevectors yet, to see if alignment can be
>> handled easily there.
>
> I agree that we=E2=80=99d need some sort of annotation to specify the ali=
gnment
> of literals, but adding read syntax for that scares me somewhat.  What
> do people think?

I agree.  The read syntax for vector-ish types in guile is already
large enough.  If alignment is important then use a procedural
constructor and query.

Alignment information not need to be printed with the default
representation (read syntax), we dont also print the storage address,
etc..

Regards
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 13 Jun 2013 13:31:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 13 09:31:10 2013
Received: from localhost ([127.0.0.1]:41102 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Un7cL-0004rJ-Df
	for submit <at> debbugs.gnu.org; Thu, 13 Jun 2013 09:31:09 -0400
Received: from hera.aquilenet.fr ([141.255.128.1]:48651)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@HIDDEN>) id 1Un7cH-0004r2-TK
 for 14599 <at> debbugs.gnu.org; Thu, 13 Jun 2013 09:31:06 -0400
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id 51D5AD51;
 Thu, 13 Jun 2013 15:31:02 +0200 (CEST)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id BdbFYUQFFmhO; Thu, 13 Jun 2013 15:31:02 +0200 (CEST)
Received: from pluto (reverse-83.fdn.fr [80.67.176.83])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id C582BAEC;
 Thu, 13 Jun 2013 15:31:01 +0200 (CEST)
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Jan Schukat <shookie@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
 <51B8E4B7.9060306@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 25 Prairial an 221 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0xEA52ECF4
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA  D15D 77DD 95E2 EA52 ECF4
X-OS: x86_64-unknown-linux-gnu
Date: Thu, 13 Jun 2013 15:31:01 +0200
In-Reply-To: <51B8E4B7.9060306@HIDDEN> (Jan Schukat's message of "Wed, 12
 Jun 2013 23:14:31 +0200")
Message-ID: <874nd2gmsq.fsf@HIDDEN>
User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

Hi,

Thanks for sharing your thoughts on this!

Jan Schukat <shookie@HIDDEN> skribis:

> Extending the programming api to support this is a bit more tricky. I
> guess most straightforward and backward compatible would be to just at
> a set of make-aligned-*vector and aligned-*vector and
> *->aligned-*vector functions and their scm_* versions with an
> additional alignment parameter. Optional alignment parameters on the
> old functions could be nice too, but I guess that is just asking for
> compatibility trouble.

What about something like =E2=80=98make-aligned-bytevector=E2=80=99?  This =
should be
enough, since bytevectors can be accessed through the SRFI-4 API (info
"(guile) Bytevectors as Uniform Arrays").  (And it can already be
implemented in Scheme, as I showed previously.)

> The other question is the read syntax (one of the primary reasons I'm
> doing all this). If alignment is something that should be preserved in
> the permanent representation, you also need to store it in the flags,
> since the content pointer can be aligned by coincidence. I haven't
> looked at the compiling of bytevectors yet, to see if alignment can be
> handled easily there.

I agree that we=E2=80=99d need some sort of annotation to specify the align=
ment
of literals, but adding read syntax for that scares me somewhat.  What
do people think?

On the compilation side, I think alignment will be more easily handled
in 2.2 (current =E2=80=98master=E2=80=99), which uses ELF (think of GCC=E2=
=80=99s =E2=80=98alignment=E2=80=99
attribute.)

None of this really seems doable in the 2.0 stable series.  So for now
you=E2=80=99d have to resort to one of the options discussed.

WDYT?

Thanks,
Ludo=E2=80=99.
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 13 Jun 2013 07:09:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 13 03:09:16 2013
Received: from localhost ([127.0.0.1]:40613 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Un1el-0007WU-Co
	for submit <at> debbugs.gnu.org; Thu, 13 Jun 2013 03:09:15 -0400
Received: from mout.web.de ([212.227.17.12]:64950)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>) id 1Un1ef-0007WD-7Z
 for 14599 <at> debbugs.gnu.org; Thu, 13 Jun 2013 03:09:13 -0400
Received: from [192.168.0.27] ([85.177.95.144]) by smtp.web.de (mrweb103) with
 ESMTPSA (Nemesis) id 0Lopa3-1U7e4x31B5-00fxjZ;
 Thu, 13 Jun 2013 09:07:45 +0200
Message-ID: <51B96FBD.7090100@HIDDEN>
Date: Thu, 13 Jun 2013 09:07:41 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: Andy Wingo <wingo@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87mwqv83qj.fsf@HIDDEN>
In-Reply-To: <87mwqv83qj.fsf@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:YVwPVMvUwhXx/fhxqiYiP2+oKSlBvHdHXuHjKDRGTrC
 fqabbDG6H7hKpL35fTEqsspU6dLdRcKVb3uOuPhLspTfPtbHUV
 UzHU8SKyM/++YDexaartH7B2CWEoZue6Rho2shOqUbsu2MhpBI
 EvPeWWzsPUvqUYiRXP5cmawLqep1iJbj7LO0gDeFFfNycI6VUG
 pNmDig0ut8M+C6CGXTtRA==
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

Hello again :)


On 06/12/2013 10:37 PM, Andy Wingo wrote:
> On Wed 12 Jun 2013 15:37, Jan Schukat <shookie@HIDDEN> writes:
>
>> If you want to access native uniform vectors from c, sometimes you
>> really want guarantees about the alignment.
> 16 bytes I guess?  Guile's uniforms are 8-byte-aligned by default, as
> you probably know.

Yes, 16 bytes. But more could be useful too. And as I have stated in my 
previous mail, the first element of the vector is guaranteed to be 4 
byte aligned on 32 bit machines, because it starts directly after the 3 
word header, which is allocated at 8 byte boundaries. And yes, I have 
tested this, but should be obvious from the code too.



>
> Just wondering if there is a better default.
>
>> +#ifdef SCM_VECTOR_ALIGN
>> +      contents = scm_gc_malloc_pointerless (SCM_BYTEVECTOR_HEADER_BYTES
>> + c_len + SCM_VECTOR_ALIGN,
>> +                        SCM_GC_BYTEVECTOR);
>> +      ret = PTR2SCM (contents);
>> +      contents += SCM_BYTEVECTOR_HEADER_BYTES;
>> +      contents += (addr + (SCM_VECTOR_ALIGN - 1)) & -SCM_VECTOR_ALIGN;
>> +#else
>>         contents = scm_gc_malloc_pointerless (SCM_BYTEVECTOR_HEADER_BYTES
>> + c_len,
>>                           SCM_GC_BYTEVECTOR);
>>         ret = PTR2SCM (contents);
>>         contents += SCM_BYTEVECTOR_HEADER_BYTES;
>> +#endif
> This is somewhat dangerous, as you could lose the pointer to the start,
> and then the contents get collected.
>
> I guess this can be fixed in master, if you set the "holder" field on a
> bytevector to the actual memory that you allocate.
>

Don't really understand the danger here, isn't this allocated as a whole 
block and only collected as a whole block too? What am I missing?

Having the arrays aligned according to their type by default could be a 
nice option, i.e. a word of padding for long and doubles on 32 bit 
machines, and then also introducing a new 16byte simd128 and 32 byte 
simd256 type id and their respective creation functions.


Regards

Jan Schukat
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 12 Jun 2013 21:14:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 12 17:14:46 2013
Received: from localhost ([127.0.0.1]:59537 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UmsNR-0000aG-RX
	for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 17:14:46 -0400
Received: from mout.web.de ([212.227.15.14]:50233)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>) id 1UmsNP-0000Zj-Aq
 for 14599 <at> debbugs.gnu.org; Wed, 12 Jun 2013 17:14:44 -0400
Received: from [192.168.0.27] ([92.231.241.64]) by smtp.web.de (mrweb101) with
 ESMTPSA (Nemesis) id 0MFL2m-1UaksY0i6G-00ELVU;
 Wed, 12 Jun 2013 23:14:34 +0200
Message-ID: <51B8E4B7.9060306@HIDDEN>
Date: Wed, 12 Jun 2013 23:14:31 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: =?UTF-8?B?THVkb3ZpYyBDb3VydMOocw==?= <ludo@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
In-Reply-To: <87a9mvide2.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:N+BnkWOeeGDGkuE2N6ogBXUwfaEA81TDdLTBxg3bL0qDOUrN9ee
 /VYNiaj7bUFhLG7Szbivf4fFmRzefz2mEFseBSZLws6yQuROYpQrKoVk0lYTnFnLb1ZFiL6
 VEdqEDxE4aKvz1PzAS+iKEjXepgvOANRMo+GIagfaraLmpYCFb5/mUMW6wOSFLDVmxUs8Gg
 Rc/LTDYbnqmtXnHcez3xA==
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

Thought a bit about it, and it would really be nice to have an aligned 
uniform vector API.

ATM all are 8 byte aligned, so you probably would want also to be able 
to have at least 16 and 32 byte alignment (intel's AVX has 256bit 
registers that better work aligned).
But even 64 and and more could be useful for cache line alignment, 
although that would require this to be a separate alignment, because the 
benefits of cache line alignment are kind of defeated if the header is 
in a different cache line.

So I guess just one alignment, namely that of the first element is 
feasible without wasting whole cache lines. If you really need that you 
can still use the take_*vector functions, and it's pretty rare to do 
such things anyway. But being able to control the alignment of the first 
element allows you to properly use simd instructions on those vectors.

You don't even really need any more space to store alignment 
information, since that can be directly inferred from the bytevector 
content pointer, although the bytevector flags still have more than 
enough space to store it.

Extending the programming api to support this is a bit more tricky. I 
guess most straightforward and backward compatible would be to just at a 
set of make-aligned-*vector and aligned-*vector and *->aligned-*vector 
functions and their scm_* versions with an additional alignment 
parameter. Optional alignment parameters on the old functions could be 
nice too, but I guess that is just asking for compatibility trouble.

The other question is the read syntax (one of the primary reasons I'm 
doing all this). If alignment is something that should be preserved in 
the permanent representation, you also need to store it in the flags, 
since the content pointer can be aligned by coincidence. I haven't 
looked at the compiling of bytevectors yet, to see if alignment can be 
handled easily there.

As for the text representation, I think the simplest way is to add 
another reserved character with the alignment number that works for 
uniform vectors and arrays like #vu8>8(1 2 3 4 5 6) to have the first 
element at 8byte alignment (right now the allocation pretty much ensures 
4 byte alignment of the first element on 32 bit machines and 8 byte at 
64bit machines, because gc_malloc returns 8byte aligned blocks, but the 
array starts at cell word 3. Any 64 bit type vector like double and long 
is already guaranteed to be misaligned on 32 bit platforms. Which would 
be even more unfortunate on linux x32 abi systems that uses efficient 64 
bit ints with 32 bit pointers, but cell size is determined by pointer size.

Or to construct simd 4 element arrays #2f32:2:4>16((1 2 3 4)(1 2 3 4)). 
Maybe even have a default alignment of 16 when you just use > without a 
number so #2f32:2:4>((1 2 3 4)(1 2 3 4)) is the same thing. Or even more 
convenient #m128((1 2 3 4)(1.0 1.0 1.0 1.0) (2.0 2.0)) where you can 
freely mix the underlying types and the size of the elements is inferred 
by the amount of them in each group.


So if there is interest for something like this in the main guile, I 
will make the patches. If not, I'll just stick to my crude hack for now 
and see if I need the full shebang :).


Regards

Jan Schukat


On 06/12/2013 04:59 PM, Ludovic Courtès wrote:
> severity 14599 wishlist
> thanks
>
> Hi!
>
> Jan Schukat <shookie@HIDDEN> skribis:
>
>> If you want to access native uniform vectors from c, sometimes you
>> really want guarantees about the alignment.
> [...]
>
>> This isn't necessarily true for vectors created from pre-existing
>> buffers (the take_*vector functions), but there you have control over
>> the pointer you pass, so you can make it true if needed.
>>
>> So if there is interest, maybe this could be integrated into the build
>> system as a configuration like this:
>>
>>
>> --- libguile/bytevectors.c    2013-04-11 02:16:30.000000000 +0200
>> +++ bytevectors.c    2013-06-12 14:45:16.000000000 +0200
>> @@ -223,10 +223,18 @@
>>
>>         c_len = len * (scm_i_array_element_type_sizes[element_type] / 8);
>>
>> +#ifdef SCM_VECTOR_ALIGN
>> +      contents = scm_gc_malloc_pointerless
>> (SCM_BYTEVECTOR_HEADER_BYTES + c_len + SCM_VECTOR_ALIGN,
>> +                        SCM_GC_BYTEVECTOR);
>> +      ret = PTR2SCM (contents);
>> +      contents += SCM_BYTEVECTOR_HEADER_BYTES;
>> +      contents += (addr + (SCM_VECTOR_ALIGN - 1)) & -SCM_VECTOR_ALIGN;
>> +#else
>>         contents = scm_gc_malloc_pointerless
>> (SCM_BYTEVECTOR_HEADER_BYTES + c_len,
>>                           SCM_GC_BYTEVECTOR);
>>         ret = PTR2SCM (contents);
>>         contents += SCM_BYTEVECTOR_HEADER_BYTES;
>> +#endif
>>
>>         SCM_BYTEVECTOR_SET_LENGTH (ret, c_len);
>>         SCM_BYTEVECTOR_SET_CONTENTS (ret, contents);
> I don’t think it should be a compile-time option, because it would be
> inflexible and inconvenient.
>
> Instead, I would suggest using the scm_take_ functions if allocating
> from C, as you noted.
>
> In Scheme, I came up with the following hack:
>
> --8<---------------cut here---------------start------------->8---
> (use-modules (system foreign)
>               (rnrs bytevectors)
>               (ice-9 match))
>
> (define (memalign len alignment)
>    (let* ((b (make-bytevector (+ len alignment)))
>           (p (bytevector->pointer b))
>           (a (pointer-address p)))
>      (match (modulo a alignment)
>        (0 b)
>        (padding
>         (let ((p (make-pointer (+ a (- alignment padding)))))
>           ;; XXX: Keep a weak reference to B or it can be collected
>           ;; behind our back.
>           (pointer->bytevector p len))))))
> --8<---------------cut here---------------end--------------->8---
>
> Not particularly elegant, but it does the job.  ;-)
>
> Do you think there’s additional support that should be provided?
>
> Thanks,
> Ludo’.
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 12 Jun 2013 20:38:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 12 16:38:11 2013
Received: from localhost ([127.0.0.1]:59452 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Umro2-0007p3-O1
	for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 16:38:11 -0400
Received: from a-pb-sasl-quonix.pobox.com ([208.72.237.25]:61141
 helo=sasl.smtp.pobox.com) by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <wingo@HIDDEN>) id 1Umrny-0007ol-7K
 for 14599 <at> debbugs.gnu.org; Wed, 12 Jun 2013 16:38:07 -0400
Received: from sasl.smtp.pobox.com (unknown [127.0.0.1])
 by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTP id 19D6BCEB1;
 Wed, 12 Jun 2013 16:38:02 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc
 :subject:references:date:in-reply-to:message-id:mime-version
 :content-type; s=sasl; bh=J3cgGxl1CUO76TCaYm55OpsQRtQ=; b=igeg24
 MIgwrwf46MQYQlNn3fHdAAPDum1UY0/Nf2a/55tEyx57MF1Q8iU+NZeqiu6JJ2o3
 QP8ijjkU+ujKw9T78tsovuR5l5qpO3GyyBpoXR/DpTyHDSosQ2vkqWXMRxfKgotJ
 H9+I5Kkv8dnrS5S2cJoUHC8ROE7j++ZFLPAu0=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc
 :subject:references:date:in-reply-to:message-id:mime-version
 :content-type; q=dns; s=sasl; b=X+7tUD4OKkduIK3QSaMhqEi7rAvA6OHi
 N+Z9FCsnJJGY0sxHjXAWBJRCtBqVV10iKkU9huXNvaPm/5zL7MPh/KQMynA6rvQw
 tXERahTXFqhp3Lr8jL89bbXPySd48R9eIcTLzsSync8oudhb4aCsdKvsBMFMeXo2
 aDZpKn5Ga2M=
Received: from a-pb-sasl-quonix.pobox.com (unknown [127.0.0.1])
 by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTP id 96D5BCEAE;
 Wed, 12 Jun 2013 16:37:46 -0400 (EDT)
Received: from badger (unknown [88.160.190.192])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTPSA id 14EE6CEAB;
 Wed, 12 Jun 2013 16:37:43 -0400 (EDT)
From: Andy Wingo <wingo@HIDDEN>
To: Jan Schukat <shookie@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN>
Date: Wed, 12 Jun 2013 22:37:40 +0200
In-Reply-To: <51B87998.9060402@HIDDEN> (Jan Schukat's message of "Wed, 12
 Jun 2013 15:37:28 +0200")
Message-ID: <87mwqv83qj.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Pobox-Relay-ID: EF474BF8-D39F-11E2-A06E-9F710E5B5709-02397024!a-pb-sasl-quonix.pobox.com
X-Spam-Score: -0.2 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.2 (/)

On Wed 12 Jun 2013 15:37, Jan Schukat <shookie@HIDDEN> writes:

> If you want to access native uniform vectors from c, sometimes you
> really want guarantees about the alignment.

16 bytes I guess?  Guile's uniforms are 8-byte-aligned by default, as
you probably know.

Just wondering if there is a better default.

> +#ifdef SCM_VECTOR_ALIGN
> +      contents = scm_gc_malloc_pointerless (SCM_BYTEVECTOR_HEADER_BYTES
> + c_len + SCM_VECTOR_ALIGN,
> +                        SCM_GC_BYTEVECTOR);
> +      ret = PTR2SCM (contents);
> +      contents += SCM_BYTEVECTOR_HEADER_BYTES;
> +      contents += (addr + (SCM_VECTOR_ALIGN - 1)) & -SCM_VECTOR_ALIGN;
> +#else
>        contents = scm_gc_malloc_pointerless (SCM_BYTEVECTOR_HEADER_BYTES
> + c_len,
>                          SCM_GC_BYTEVECTOR);
>        ret = PTR2SCM (contents);
>        contents += SCM_BYTEVECTOR_HEADER_BYTES;
> +#endif

This is somewhat dangerous, as you could lose the pointer to the start,
and then the contents get collected.

I guess this can be fixed in master, if you set the "holder" field on a
bytevector to the actual memory that you allocate.

Andy
-- 
http://wingolog.org/
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 12 Jun 2013 15:32:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 12 11:32:48 2013
Received: from localhost ([127.0.0.1]:59012 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Umn2V-0002j1-P4
	for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 11:32:47 -0400
Received: from mout.web.de ([212.227.17.11]:51778)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>)
 id 1Umn2U-0002ia-E9; Wed, 12 Jun 2013 11:32:47 -0400
Received: from [192.168.0.27] ([92.231.241.64]) by smtp.web.de (mrweb102) with
 ESMTPSA (Nemesis) id 0MUWBb-1UwWMo2RjX-00RLHh;
 Wed, 12 Jun 2013 17:32:38 +0200
Message-ID: <51B89493.8050805@HIDDEN>
Date: Wed, 12 Jun 2013 17:32:35 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: =?UTF-8?B?THVkb3ZpYyBDb3VydMOocw==?= <ludo@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN> <87a9mvide2.fsf@HIDDEN>
In-Reply-To: <87a9mvide2.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:l2JJ6U+EC9Z4WwLBhfzyAdhDP4T9/6PuwJn16C4y8MatdqHHMts
 7KB7nH5VroTVi2udPy3l5B5Z21uJhBVnaPBJZDZ2tbb05RAtDvLEcJR4Nc/EClmxtvau0ZG
 bwTQpEqxi84k6afEOBYH+jS74QvYaWVxz6r+mHVr4pEexyPf5NtuYHXAPYvu+A6+ubW2UIi
 mdTbLT15AOIfEexBMA2Vw==
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org, request <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

On 06/12/2013 04:59 PM, Ludovic Courtès wrote:
> Instead, I would suggest using the scm_take_ functions if allocating
> from C, as you noted.

The whole point of me doing this is so I can use lisp files to define aligned data (and using lisp as a flexible text data format that can be compiled is the primary reason I use guile).

So either I make normal vectors aligned, or I define a whole new set of datatypes of aligned vectors with their own read syntax.

The former I just did today, the latter would take me probably a quite a bit of time to do properly. So, I'm gonna keep using my compile time option for now, and probably write a module of aligned vectors at some point in the future when I have more of an understanding what exactly the requirements for something like this would be.

Jan
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.
Severity set to 'wishlist' from 'normal' Request was from ludo@HIDDEN (Ludovic Courtès) to control <at> debbugs.gnu.org. Full text available.

Message received at 14599 <at> debbugs.gnu.org:


Received: (at 14599) by debbugs.gnu.org; 12 Jun 2013 14:59:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 12 10:59:07 2013
Received: from localhost ([127.0.0.1]:58986 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UmmVu-0000I9-Il
	for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 10:59:06 -0400
Received: from hera.aquilenet.fr ([141.255.128.1]:46805)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@HIDDEN>)
 id 1UmmVr-0000Hc-L7; Wed, 12 Jun 2013 10:59:05 -0400
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id 28E87CDD;
 Wed, 12 Jun 2013 16:59:02 +0200 (CEST)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id OZr31Gj4XSwm; Wed, 12 Jun 2013 16:59:02 +0200 (CEST)
Received: from pluto (eduroam-3-182.epfl.ch [192.33.205.182])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id D0A9ECC1;
 Wed, 12 Jun 2013 16:59:01 +0200 (CEST)
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Jan Schukat <shookie@HIDDEN>
Subject: Re: bug#14599: An option to make vector allocation aligned
References: <51B87998.9060402@HIDDEN>
Date: Wed, 12 Jun 2013 16:59:01 +0200
In-Reply-To: <51B87998.9060402@HIDDEN> (Jan Schukat's message of "Wed, 12
 Jun 2013 15:37:28 +0200")
Message-ID: <87a9mvide2.fsf@HIDDEN>
User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 14599
Cc: 14599 <at> debbugs.gnu.org, request <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

severity 14599 wishlist
thanks

Hi!

Jan Schukat <shookie@HIDDEN> skribis:

> If you want to access native uniform vectors from c, sometimes you
> really want guarantees about the alignment.

[...]

> This isn't necessarily true for vectors created from pre-existing
> buffers (the take_*vector functions), but there you have control over
> the pointer you pass, so you can make it true if needed.
>
> So if there is interest, maybe this could be integrated into the build
> system as a configuration like this:
>
>
> --- libguile/bytevectors.c    2013-04-11 02:16:30.000000000 +0200
> +++ bytevectors.c    2013-06-12 14:45:16.000000000 +0200
> @@ -223,10 +223,18 @@
>
>        c_len =3D len * (scm_i_array_element_type_sizes[element_type] / 8);
>
> +#ifdef SCM_VECTOR_ALIGN
> +      contents =3D scm_gc_malloc_pointerless
> (SCM_BYTEVECTOR_HEADER_BYTES + c_len + SCM_VECTOR_ALIGN,
> +                        SCM_GC_BYTEVECTOR);
> +      ret =3D PTR2SCM (contents);
> +      contents +=3D SCM_BYTEVECTOR_HEADER_BYTES;
> +      contents +=3D (addr + (SCM_VECTOR_ALIGN - 1)) & -SCM_VECTOR_ALIGN;
> +#else
>        contents =3D scm_gc_malloc_pointerless
> (SCM_BYTEVECTOR_HEADER_BYTES + c_len,
>                          SCM_GC_BYTEVECTOR);
>        ret =3D PTR2SCM (contents);
>        contents +=3D SCM_BYTEVECTOR_HEADER_BYTES;
> +#endif
>
>        SCM_BYTEVECTOR_SET_LENGTH (ret, c_len);
>        SCM_BYTEVECTOR_SET_CONTENTS (ret, contents);

I don=E2=80=99t think it should be a compile-time option, because it would =
be
inflexible and inconvenient.

Instead, I would suggest using the scm_take_ functions if allocating
from C, as you noted.

In Scheme, I came up with the following hack:

--8<---------------cut here---------------start------------->8---
(use-modules (system foreign)
             (rnrs bytevectors)
             (ice-9 match))

(define (memalign len alignment)
  (let* ((b (make-bytevector (+ len alignment)))
         (p (bytevector->pointer b))
         (a (pointer-address p)))
    (match (modulo a alignment)
      (0 b)
      (padding
       (let ((p (make-pointer (+ a (- alignment padding)))))
         ;; XXX: Keep a weak reference to B or it can be collected
         ;; behind our back.
         (pointer->bytevector p len))))))
--8<---------------cut here---------------end--------------->8---

Not particularly elegant, but it does the job.  ;-)

Do you think there=E2=80=99s additional support that should be provided?

Thanks,
Ludo=E2=80=99.
Information forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 12 Jun 2013 13:38:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 12 09:38:01 2013
Received: from localhost ([127.0.0.1]:58457 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UmlFR-00051r-CL
	for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 09:38:01 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41785)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <shookie@HIDDEN>) id 1UmlFP-00051e-AM
 for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 09:37:59 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <shookie@HIDDEN>) id 1UmlFC-0001KZ-Ew
 for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 09:37:53 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:51099)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <shookie@HIDDEN>) id 1UmlFC-0001KV-B3
 for submit <at> debbugs.gnu.org; Wed, 12 Jun 2013 09:37:46 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43437)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <shookie@HIDDEN>) id 1UmlF5-0004JS-Nt
 for bug-guile@HIDDEN; Wed, 12 Jun 2013 09:37:46 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <shookie@HIDDEN>) id 1UmlEz-0001GO-9z
 for bug-guile@HIDDEN; Wed, 12 Jun 2013 09:37:39 -0400
Received: from mout.web.de ([212.227.17.11]:58857)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <shookie@HIDDEN>) id 1UmlEy-0001FN-Ve
 for bug-guile@HIDDEN; Wed, 12 Jun 2013 09:37:33 -0400
Received: from [192.168.0.27] ([92.231.241.64]) by smtp.web.de (mrweb002) with
 ESMTPSA (Nemesis) id 0M9XbT-1UdeIn2j6N-00CzTq for
 <bug-guile@HIDDEN>; Wed, 12 Jun 2013 15:37:30 +0200
Message-ID: <51B87998.9060402@HIDDEN>
Date: Wed, 12 Jun 2013 15:37:28 +0200
From: Jan Schukat <shookie@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:17.0) Gecko/20130510 Thunderbird/17.0.6
MIME-Version: 1.0
To: bug-guile@HIDDEN
Subject: An option to make vector allocation aligned
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K0:+vlz2YM+i8/Gr0HN+FFArYbOdMDW20D/WO6SF50MBKkteFhujci
 bRV/XcHMhM3iDZgPIQNmFDSmdOhw4EgCKUGLfppOJXsj41OQuvr7vOJQ4Tzda4LGe0bKZvi
 BefO0sAHVbohNM9YMw7n1sdmwhb9oPvroHpi9cqneNQ0ozlve9akULn+5weAWSe9YeIW4a+
 Lkr81WTak+zJcamuSvGeA==
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic]
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Hello,

If you want to access native uniform vectors from c, sometimes you 
really want guarantees about the alignment.

Fortunately the the (byte)vector format and allocation makes that pretty 
easy to implement: just add a little padding between the header and the 
actual data.

So for my own project, this is what I'm doing, and there shouldn't be 
much of a memory impact unless there are tons of small vectors used, 
which isn't very lispy anyway.

This isn't necessarily true for vectors created from pre-existing 
buffers (the take_*vector functions), but there you have control over 
the pointer you pass, so you can make it true if needed.

So if there is interest, maybe this could be integrated into the build 
system as a configuration like this:


--- libguile/bytevectors.c    2013-04-11 02:16:30.000000000 +0200
+++ bytevectors.c    2013-06-12 14:45:16.000000000 +0200
@@ -223,10 +223,18 @@

        c_len = len * (scm_i_array_element_type_sizes[element_type] / 8);

+#ifdef SCM_VECTOR_ALIGN
+      contents = scm_gc_malloc_pointerless (SCM_BYTEVECTOR_HEADER_BYTES 
+ c_len + SCM_VECTOR_ALIGN,
+                        SCM_GC_BYTEVECTOR);
+      ret = PTR2SCM (contents);
+      contents += SCM_BYTEVECTOR_HEADER_BYTES;
+      contents += (addr + (SCM_VECTOR_ALIGN - 1)) & -SCM_VECTOR_ALIGN;
+#else
        contents = scm_gc_malloc_pointerless 
(SCM_BYTEVECTOR_HEADER_BYTES + c_len,
                          SCM_GC_BYTEVECTOR);
        ret = PTR2SCM (contents);
        contents += SCM_BYTEVECTOR_HEADER_BYTES;
+#endif

        SCM_BYTEVECTOR_SET_LENGTH (ret, c_len);
        SCM_BYTEVECTOR_SET_CONTENTS (ret, contents);


It could even be possible to make the alignment a run-time decision, but 
for that the api and read syntax for vectors need to be extended. Which 
could be worthwhile ...

Apart from that, I see there are issues with the native mingw builds 
again, which I haven't noticed earlier since I primarily develop on 
linux, but I can reproduce the problem shown in #14361.


Regards

Jan Schukat
Acknowledgement sent to Jan Schukat <shookie@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guile@HIDDEN. Full text available.
Report forwarded to bug-guile@HIDDEN:
bug#14599; Package guile. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.