GNU bug report logs - #6366
join can't join on numeric fields

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: coreutils; Severity: wishlist; Reported by: Alex Shinn <alexshinn@HIDDEN>; Keywords: patch; merged with #10924, #12264; dated Mon, 7 Jun 2010 05:24:02 UTC; Maintainer for coreutils is bug-coreutils@HIDDEN.
Severity set to 'wishlist' from 'normal' Request was from Assaf Gordon <assafgordon@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 9 Oct 2018 20:19:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 09 16:19:16 2018
Received: from localhost ([127.0.0.1]:42596 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1g9yTL-00027r-Ra
	for submit <at> debbugs.gnu.org; Tue, 09 Oct 2018 16:19:16 -0400
Received: from mail-it1-f171.google.com ([209.85.166.171]:54237)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <assafgordon@HIDDEN>)
 id 1g9yTJ-00027T-FV; Tue, 09 Oct 2018 16:19:13 -0400
Received: by mail-it1-f171.google.com with SMTP id q70-v6so4726298itb.3;
 Tue, 09 Oct 2018 13:19:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=iamIcN/LH25OyYYrSviMeEYdVcKQNwIqZATB8YbsbFc=;
 b=pOD44LRxsQwF95w+fYXzEZCDrFQTGV1IR1FeLwrCYNsafonTYr/cOfpQErZqnN6hu+
 lEvBj7PNFojlWqY8G/TKBnMuDa2/s7SdtGt4PFAU7xQvtlB1GLM+WpxyKaIuff7BC8zj
 oWy6R92lotktbYD101fjYDmQDUPM/7WZPt8bJyPSq0HB1z/bVilOsrx6+Ib33iSFSab/
 gbNUiSUcAPPIlktPQm7038lKW+6y2XazxvgkTV7gIDHVDUXD4G5GMU8yz3rG1NW9Q0kt
 mHx94Nbp6Te1r/m4XeisiZYyx3e6aU1lppaYm+P1VdfD5w50Pgl5kJvazcVNvKz0JQBp
 0rJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:subject:to:cc:references:from:message-id:date
 :user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=iamIcN/LH25OyYYrSviMeEYdVcKQNwIqZATB8YbsbFc=;
 b=Cpj4v7FdLTzl0SCCbQqAXKWFhwb/QiCe0GJdTU+QcJF1iZjUcFtHPxqTfNar/N/aMK
 5iEdjEnqHYvc/HG9eM4LSqBKBhPnsjcE9GD6McyArnUhQLpAzllIFIcGJVxmtuHQK7ug
 r3FfxW9+JxBUypsGY4PBaGDaodYVaAZiU26nhD/nCNMVbBTm2+bObD1YEluQ3EsHt7mn
 tHBc5rmRFoFQ572R52onhGDFDKGti5fWGEIsgfUNWuK7Yo8KHnVYgMXXgCiRNKbNfjZK
 QIxrvxXGj6hsdtvHeou3typHjIvGNlvtdQ8ZsQlStvWzHbKnSVkneZQdkv98G3hfRon7
 2HCQ==
X-Gm-Message-State: ABuFfohZNqhkhJ43B/1Gv+T+BpTA32DbYtJagNyI+6tSeJGwdk9nDRnr
 zioZZp2Ct8Sx/G57lZ4iC8dG7Xanm5A=
X-Google-Smtp-Source: ACcGV60uHp2COComao45KnpHpjgl3cnc98G3+YJk/SDcQOCBOnsEUK8dDhOZqB1e7B9a2UIwCxyxjA==
X-Received: by 2002:a24:95c4:: with SMTP id
 m187-v6mr3028073itd.83.1539116347354; 
 Tue, 09 Oct 2018 13:19:07 -0700 (PDT)
Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38])
 by smtp.googlemail.com with ESMTPSA id
 j75-v6sm6652840itj.33.2018.10.09.13.19.05
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 09 Oct 2018 13:19:05 -0700 (PDT)
Subject: Re: bug#6366: comm: use numeric sort (optionally)
To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>
References: <87sjbeisvv.fsf@HIDDEN> <50355695.2030201@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <42953a1d-3f9a-329b-c0bd-6fa37be379af@HIDDEN>
Date: Tue, 9 Oct 2018 14:19:04 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <50355695.2030201@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 6366
Cc: 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

severity 6366 wishlist
stop

(triaging old bugs)

Hello,

On 22/08/12 04:00 PM, Pádraig Brady wrote:
> 
> On 08/22/2012 09:17 PM, Sam Steingold wrote:
>> I have a file of numbers (integers IDs) which are sorted numerically,
>> but comm complains that they are not.
>> I suggest that comm accept "-n" option (like sort) to use numeric sort.
>> Thanks.
> 
> Yes.
> comm, join, uniq really should support the same field selection
> and comparision flags as sort.
> 
> There are already bugs for that:
> http://bugs.gnu.org/5832
> http://bugs.gnu.org/6366

There is an on-going (though somewhat dormant) effort to
consolidate the key comparison of uniq/join/sort into one common
module:

   https://lists.gnu.org/r/coreutils/2013-02/msg00087.html
   https://lists.gnu.org/r/coreutils/2016-04/msg00063.html


As such, I'm marking this as a wishlist item, and hopefully we'll get
to it sooner or later...

regards,
  - assaf






Information forwarded to bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.
Forcibly Merged 6366 10924 12264. Request was from era eriksson <era@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Forcibly Merged 6366 12264. Request was from Pádraig Brady <P@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 22 Aug 2012 22:01:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 22 18:01:26 2012
Received: from localhost ([127.0.0.1]:43872 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1T4IzO-00039U-Mo
	for submit <at> debbugs.gnu.org; Wed, 22 Aug 2012 18:01:26 -0400
Received: from mx1.redhat.com ([209.132.183.28]:63825)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <P@HIDDEN>)
	id 1T4IzM-00039J-2X; Wed, 22 Aug 2012 18:01:25 -0400
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q7MM0uKK027992
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 22 Aug 2012 18:00:56 -0400
Received: from [10.36.116.69] (ovpn-116-69.ams2.redhat.com [10.36.116.69])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q7MM0rM8004906
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Wed, 22 Aug 2012 18:00:55 -0400
Message-ID: <50355695.2030201@HIDDEN>
Date: Wed, 22 Aug 2012 23:00:53 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: sds@HIDDEN
Subject: Re: comm: use numeric sort (optionally)
References: <87sjbeisvv.fsf@HIDDEN>
In-Reply-To: <87sjbeisvv.fsf@HIDDEN>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q7MM0uKK027992
X-Spam-Score: -6.9 (------)
X-Debbugs-Envelope-To: 6366
Cc: coreutils@HIDDEN, 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -6.9 (------)

unarchive 6366
stop

On 08/22/2012 09:17 PM, Sam Steingold wrote:
> Hi,
> I have a file of numbers (integers IDs) which are sorted numerically,
> but comm complains that they are not.
> I suggest that comm accept "-n" option (like sort) to use numeric sort.
> Thanks.

Yes.
comm, join, uniq really should support the same field selection
and comparision flags as sort.

There are already bugs for that:
http://bugs.gnu.org/5832
http://bugs.gnu.org/6366

For reference, the following examples show
expected an unexpected behavior respectively:

$ comm --nocheck <(printf "%s\n" a b c j k) <(printf "%s\n" a b d e j)
		a
		b
c
	d
	e
		j
k

$ comm --nocheck <(printf "%s\n" 1 2 3 10 11) <(printf "%s\n" 1 2 5 6 10)
		1
		2
3
10
11
	5
	6
	10

cheers,
P=E1draig.




Information forwarded to bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 21 Mar 2012 16:40:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 21 12:40:08 2012
Received: from localhost ([127.0.0.1]:59835 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1SAOZy-0001Lt-Gn
	for submit <at> debbugs.gnu.org; Wed, 21 Mar 2012 12:40:07 -0400
Received: from mail-ob0-f172.google.com ([209.85.214.172]:64770)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <drewfrank@HIDDEN>) id 1SAEfv-0001So-GL
	for 6366 <at> debbugs.gnu.org; Wed, 21 Mar 2012 02:05:37 -0400
Received: by obbtb4 with SMTP id tb4so472477obb.3
	for <6366 <at> debbugs.gnu.org>; Tue, 20 Mar 2012 22:34:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:sender:from:date:x-google-sender-auth:message-id
	:subject:to:content-type;
	bh=6L9q/MGcoHM5AWQtRaImz7o6IC13lVavUZiZmc+yC/E=;
	b=MyGoGo2JkrxkF9Ce9hnP6MHXXsZkYQc02zufsPjQNT4ym/LCzyMN2AKEbkjjfsxUmz
	625UUnJ+6upgeUUq0cb4pmTmZCpNDDcQ6ck5Qops1IEAvDEuj9UfHVNXg5J41w060xZd
	1RKWu3sfCtHqAOzHzN8V4igpQ/a99kc9kZA2zPYEI9BO2EKm9Uu/Y9X2sWWiUGlQx5B0
	i/9chpZYQ1KiUwLuZaS8fnPAqyv7/w2CBbJyI4gdEMctm7ZeiM794R+YYb3XEqOrONgK
	NREBtFd8SSgoJlRXV7ga/kuEfHLNdpV3WK9pSo9iwJSSF8NU7oEv/qCiQo65oT3JXF3+
	FnJQ==
Received: by 10.182.134.97 with SMTP id pj1mr3164516obb.2.1332308094024; Tue,
	20 Mar 2012 22:34:54 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.113.42 with HTTP; Tue, 20 Mar 2012 22:34:38 -0700 (PDT)
From: Drew Frank <ajfrank@HIDDEN>
Date: Tue, 20 Mar 2012 22:34:38 -0700
X-Google-Sender-Auth: lbwmcUvKr7FsPdY2q2e6q08PdvA
Message-ID: <CA+pPVKv4rnXy-20zA8PhhTQZCf+ppc6aY9-eZ4Zvo32KdY22hA@HIDDEN>
Subject: join can't join on numeric fields
To: 6366 <at> debbugs.gnu.org
Content-Type: multipart/mixed; boundary=e89a8f83a645acfaf804bbba26a5
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 6366
X-Mailman-Approved-At: Wed, 21 Mar 2012 12:40:04 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

--e89a8f83a645acfaf804bbba26a5
Content-Type: multipart/alternative; boundary=e89a8f83a645acfaf304bbba26a3

--e89a8f83a645acfaf304bbba26a3
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I've attached a patch that implements both "numeric sort" (-n) and "general
numeric sort" (-g).. I copied the relevant comparison code from sort.c to
ensure consistent behavior. Should these functions be extracted and put in
a shared header file, or is it okay to duplicate the code?

In terms of testing, I confirmed that the code correctly deals with
different single-byte thousand separators on my local machine, but I wasn't
sure how to exercise that in the test suite since I can't rely on everyone
having a particular locale available. How should that be handled?

I also updated the relevant docs.

Thanks,
Drew

P.S. I sent a previous version of the patch to the wrong address and
accidentally initiated a new bug thread (#10924). This patch supersedes the
previous.

--e89a8f83a645acfaf304bbba26a3
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,<br><br>I&#39;ve attached a patch that implements both &quot;numeric sor=
t&quot; (-n) and &quot;general numeric sort&quot; (-g).. I copied the relev=
ant comparison code from sort.c to ensure consistent behavior. Should these=
 functions be extracted and put in a shared header file, or is it okay to d=
uplicate the code?<br>




<br>In terms of testing, I confirmed that the code correctly deals with dif=
ferent single-byte thousand separators on my local machine, but I wasn&#39;=
t sure how to exercise that in the test suite since I can&#39;t rely on eve=
ryone having a particular locale available. How should that be handled?<br>

<br>I also updated the relevant docs.<br>


<br>Thanks,<br>Drew<br><br>P.S. I sent a previous version of the patch to t=
he wrong address and accidentally initiated a new bug thread (#10924). This=
 patch supersedes the previous.<br>

--e89a8f83a645acfaf304bbba26a3--
--e89a8f83a645acfaf804bbba26a5
Content-Type: text/x-patch; charset=US-ASCII; name="join.numeric.sort.diff"
Content-Disposition: attachment; filename="join.numeric.sort.diff"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_h01xq5wi0

RnJvbSA5OWE5YzRjN2Q3YTgyOGE5MTdlMTRmN2U1MjQxOTIxYTRkYzc3OTA5IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBEcmV3IEZyYW5rIDxkcmV3ZnJhbmtAZ21haWwuY29tPgpEYXRl
OiBUaHUsIDEgTWFyIDIwMTIgMTQ6MjQ6NDkgLTA4MDAKU3ViamVjdDogW1BBVENIXSBqb2luOiBh
ZGQgbnVtZXJpYyBzb3J0IGZlYXR1cmUgKC1nIGFuZCAtbiBmbGFncykuCgoqIHNyYy9qb2luLmM6
IGFkZCBmbGFncyBhbmQgaW1wbGVtZW50IG51bWVyaWMgY29tcGFyaXNvbiBmZWF0dXJlcy4KKiB0
ZXN0cy9taXNjL2pvaW46IGFkZCB0aHJlZSB0ZXN0cyBmb3IgbnVtZXJpY2FsbHkgc29ydGVkIGtl
eSBmaWVsZHMuCiogZG9jL2NvcmV1dGlscy50ZXhpOiBkb2N1bWVudCBuZXcgZnVuY3Rpb25hbGl0
eS4KLS0tCiBkb2MvY29yZXV0aWxzLnRleGkgfCAgIDE2ICsrKysrKwogZ251bGliICAgICAgICAg
ICAgIHwgICAgMiArLQogc3JjL2pvaW4uYyAgICAgICAgIHwgIDE1NyArKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tCiB0ZXN0cy9taXNjL2pvaW4gICAg
fCAgICA4ICsrKwogNCBmaWxlcyBjaGFuZ2VkLCAxNzggaW5zZXJ0aW9ucygrKSwgNSBkZWxldGlv
bnMoLSkKCmRpZmYgLS1naXQgYS9kb2MvY29yZXV0aWxzLnRleGkgYi9kb2MvY29yZXV0aWxzLnRl
eGkKaW5kZXggMTBiZTcxNS4uMDE3MTE4ZiAxMDA2NDQKLS0tIGEvZG9jL2NvcmV1dGlscy50ZXhp
CisrKyBiL2RvYy9jb3JldXRpbHMudGV4aQpAQCAtNTc5OCw2ICs1Nzk4LDE0IEBAIHNwZWNpZmll
ZCBmb3JtYXQuICBUaGUgaGVhZGVyIGxpbmVzIHdpbGwgbm90IGJlIGNoZWNrZWQgZm9yIG9yZGVy
aW5nIGV2ZW4gaWYKIEBvcHRpb257LS1jaGVjay1vcmRlcn0gaXMgc3BlY2lmaWVkLiAgQWxzbyBp
ZiB0aGUgaGVhZGVyIGxpbmVzIGZyb20gZWFjaCBmaWxlCiBkbyBub3QgbWF0Y2gsIHRoZSBoZWFk
aW5nIGZpZWxkcyBmcm9tIHRoZSBmaXJzdCBmaWxlIHdpbGwgYmUgdXNlZC4KIAorQGl0ZW0gLWcK
K0BpdGVteCAtLWdlbmVyYWwtbnVtZXJpYy1zb3J0CitAb3BpbmRleCAtZworQG9waW5kZXggLS1n
ZW5lcmFsLW51bWVyaWMtc29ydAorQ29tcHV0ZSBnZW5lcmFsIG51bWVyaWNhbCB2YWx1ZSB3aGVu
IGNvbXBhcmluZyBrZXlzLgorV2l0aCB0aGlzIG9wdGlvbiwgdGhlIGxpbmVzIG9mIHRoZSBpbnB1
dCBmaWxlcyBtdXN0IGJlIG9yZGVyZWQgaW4gdGhlIHNhbWUgd2F5LgorVXNlIEBzYW1we3NvcnQg
LWd9IHRvIHByb2R1Y2UgdGhpcyBvcmRlcmluZy4KKwogQGl0ZW0gLWkKIEBpdGVteCAtLWlnbm9y
ZS1jYXNlCiBAb3BpbmRleCAtaQpAQCAtNTgwNiw2ICs1ODE0LDE0IEBAIElnbm9yZSBkaWZmZXJl
bmNlcyBpbiBjYXNlIHdoZW4gY29tcGFyaW5nIGtleXMuCiBXaXRoIHRoaXMgb3B0aW9uLCB0aGUg
bGluZXMgb2YgdGhlIGlucHV0IGZpbGVzIG11c3QgYmUgb3JkZXJlZCBpbiB0aGUgc2FtZSB3YXku
CiBVc2UgQHNhbXB7c29ydCAtZn0gdG8gcHJvZHVjZSB0aGlzIG9yZGVyaW5nLgogCitAaXRlbSAt
bgorQGl0ZW14IC0tbnVtZXJpYy1zb3J0CitAb3BpbmRleCAtbgorQG9waW5kZXggLS1udW1lcmlj
LXNvcnQKK0NvbXB1dGUgc3RyaW5nIG51bWVyaWNhbCB2YWx1ZSB3aGVuIGNvbXBhcmluZyBrZXlz
LgorV2l0aCB0aGlzIG9wdGlvbiwgdGhlIGxpbmVzIG9mIHRoZSBpbnB1dCBmaWxlcyBtdXN0IGJl
IG9yZGVyZWQgaW4gdGhlIHNhbWUgd2F5LgorVXNlIEBzYW1we3NvcnQgLW59IHRvIHByb2R1Y2Ug
dGhpcyBvcmRlcmluZy4KKwogQGl0ZW0gLTEgQHZhcntmaWVsZH0KIEBvcGluZGV4IC0xCiBKb2lu
IG9uIGZpZWxkIEB2YXJ7ZmllbGR9IChhIHBvc2l0aXZlIGludGVnZXIpIG9mIGZpbGUgMS4KZGlm
ZiAtLWdpdCBhL2dudWxpYiBiL2dudWxpYgppbmRleCA0NzMwYzNlLi5jYmMxMWZmIDE2MDAwMAot
LS0gYS9nbnVsaWIKKysrIGIvZ251bGliCkBAIC0xICsxIEBACi1TdWJwcm9qZWN0IGNvbW1pdCA0
NzMwYzNlMzY5MmIzNDRlZmZiNzJkNDZiM2ZmOTJkYjBiZGI3OTdhCitTdWJwcm9qZWN0IGNvbW1p
dCBjYmMxMWZmMDAyMGViOWMwNGNhZWE2YjNlN2RjNGU0MjgxZGZmMWY5CmRpZmYgLS1naXQgYS9z
cmMvam9pbi5jIGIvc3JjL2pvaW4uYwppbmRleCBiOTJjMWY4Li45NWRhYzQ3IDEwMDY0NAotLS0g
YS9zcmMvam9pbi5jCisrKyBiL3NyYy9qb2luLmMKQEAgLTMwLDYgKzMwLDcgQEAKICNpbmNsdWRl
ICJtZW1jYXNlY21wLmgiCiAjaW5jbHVkZSAicXVvdGUuaCIKICNpbmNsdWRlICJzdGRpby0tLmgi
CisjaW5jbHVkZSAic3RybnVtY21wLmgiCiAjaW5jbHVkZSAieG1lbWNvbGwuaCIKICNpbmNsdWRl
ICJ4c3RydG9sLmgiCiAjaW5jbHVkZSAiYXJnbWF0Y2guaCIKQEAgLTQ3LDYgKzQ4LDE2IEBACiAg
IGIgPSB0bXA7IFwKIH0gd2hpbGUgKDApOwogCisjZGVmaW5lIFVDSEFSX0xJTSAoVUNIQVJfTUFY
ICsgMSkKKworI2lmIEhBVkVfQzk5X1NUUlRPTEQKKyMgZGVmaW5lIGxvbmdfZG91YmxlIGxvbmcg
ZG91YmxlCisjZWxzZQorIyBkZWZpbmUgbG9uZ19kb3VibGUgZG91YmxlCisjIHVuZGVmIHN0cnRv
bGQKKyMgZGVmaW5lIHN0cnRvbGQgc3RydG9kCisjZW5kaWYKKwogLyogQW4gZWxlbWVudCBvZiB0
aGUgbGlzdCBpZGVudGlmeWluZyB3aGljaCBmaWVsZHMgdG8gcHJpbnQgZm9yIGVhY2gKICAgIG91
dHB1dCBsaW5lLiAgKi8KIHN0cnVjdCBvdXRsaXN0CkBAIC0xMDAsNiArMTExLDEyIEBAIHN0YXRp
YyBjaGFyICpnX25hbWVzWzJdOwogICAgd2FudCB0byBvdmVyd3JpdGUgdGhlIHByZXZpb3VzIGJ1
ZmZlciBiZWZvcmUgd2UgY2hlY2sgb3JkZXIuICovCiBzdGF0aWMgc3RydWN0IGxpbmUgKnNwYXJl
bGluZVsyXSA9IHtOVUxMLCBOVUxMfTsKIAorLyogVGhlIHJlcHJlc2VudGF0aW9uIG9mIHRoZSBk
ZWNpbWFsIHBvaW50IGluIHRoZSBjdXJyZW50IGxvY2FsZS4gICovCitzdGF0aWMgaW50IGRlY2lt
YWxfcG9pbnQ7CisKKy8qIFRob3VzYW5kcyBzZXBhcmF0b3I7IGlmIC0xLCB0aGVuIHRoZXJlIGlz
bid0IG9uZS4gICovCitzdGF0aWMgaW50IHRob3VzYW5kc19zZXA7CisKIC8qIFRydWUgaWYgdGhl
IExDX0NPTExBVEUgbG9jYWxlIGlzIGhhcmQuICAqLwogc3RhdGljIGJvb2wgaGFyZF9MQ19DT0xM
QVRFOwogCkBAIC0xNDAsNiArMTU3LDkgQEAgc3RhdGljIHN0cnVjdCBvdXRsaXN0ICpvdXRsaXN0
X2VuZCA9ICZvdXRsaXN0X2hlYWQ7CiAgICB0YWIgY2hhcmFjdGVyIHdob3NlIHZhbHVlICh3aGVu
IGNhc3QgdG8gdW5zaWduZWQgY2hhcikgZXF1YWxzIFRBQi4gICovCiBzdGF0aWMgaW50IHRhYiA9
IC0xOwogCisvKiBUYWJsZSBvZiBibGFua3MuICAqLworc3RhdGljIGJvb2wgYmxhbmtzW1VDSEFS
X0xJTV07CisKIC8qIElmIG5vbnplcm8sIGNoZWNrIHRoYXQgdGhlIGlucHV0IGlzIGNvcnJlY3Rs
eSBvcmRlcmVkLiAqLwogc3RhdGljIGVudW0KICAgewpAQCAtMTU4LDcgKzE3OCw5IEBAIGVudW0K
IAogc3RhdGljIHN0cnVjdCBvcHRpb24gY29uc3QgbG9uZ29wdHNbXSA9CiB7CisgIHsiZ2VuZXJh
bC1udW1lcmljLXNvcnQiLCBub19hcmd1bWVudCwgTlVMTCwgJ2cnfSwKICAgeyJpZ25vcmUtY2Fz
ZSIsIG5vX2FyZ3VtZW50LCBOVUxMLCAnaSd9LAorICB7Im51bWVyaWMtc29ydCIsIG5vX2FyZ3Vt
ZW50LCBOVUxMLCAnbid9LAogICB7ImNoZWNrLW9yZGVyIiwgbm9fYXJndW1lbnQsIE5VTEwsIENI
RUNLX09SREVSX09QVElPTn0sCiAgIHsibm9jaGVjay1vcmRlciIsIG5vX2FyZ3VtZW50LCBOVUxM
LCBOT0NIRUNLX09SREVSX09QVElPTn0sCiAgIHsiaGVhZGVyIiwgbm9fYXJndW1lbnQsIE5VTEws
IEhFQURFUl9MSU5FX09QVElPTn0sCkBAIC0xNzMsNiArMTk1LDEyIEBAIHN0YXRpYyBzdHJ1Y3Qg
bGluZSB1bmlfYmxhbms7CiAvKiBJZiBub256ZXJvLCBpZ25vcmUgY2FzZSB3aGVuIGNvbXBhcmlu
ZyBqb2luIGZpZWxkcy4gICovCiBzdGF0aWMgYm9vbCBpZ25vcmVfY2FzZTsKIAorLyogSWYgbm9u
emVybywgdHJlYXQga2V5cyBhcyBudW1lcmljIHZhbHVlcy4gICovCitzdGF0aWMgYm9vbCBudW1l
cmljX3NvcnQ7CisKKy8qIElmIG5vbnplcm8sIHRyZWF0IGtleXMgYXMgZ2VuZXJhbCBudW1lcmlj
IHZhbHVlcy4gICovCitzdGF0aWMgYm9vbCBnZW5lcmFsX251bWVyaWNfc29ydDsKKwogLyogSWYg
bm9uemVybywgdHJlYXQgdGhlIGZpcnN0IGxpbmUgb2YgZWFjaCBmaWxlIGFzIGNvbHVtbiBoZWFk
ZXJzIC0KICAgIGpvaW4gdGhlbSB3aXRob3V0IGNoZWNraW5nIGZvciBvcmRlcmluZyAqLwogc3Rh
dGljIGJvb2wgam9pbl9oZWFkZXJfbGluZXM7CkBAIC0xOTgsNyArMjI2LDkgQEAgYnkgd2hpdGVz
cGFjZS4gIFdoZW4gRklMRTEgb3IgRklMRTIgKG5vdCBib3RoKSBpcyAtLCByZWFkIHN0YW5kYXJk
IGlucHV0LlxuXAogICAtZSBFTVBUWSAgICAgICAgICByZXBsYWNlIG1pc3NpbmcgaW5wdXQgZmll
bGRzIHdpdGggRU1QVFlcblwKICIpLCBzdGRvdXQpOwogICAgICAgZnB1dHMgKF8oIlwKLSAgLWks
IC0taWdub3JlLWNhc2UgIGlnbm9yZSBkaWZmZXJlbmNlcyBpbiBjYXNlIHdoZW4gY29tcGFyaW5n
IGZpZWxkc1xuXAorICAtZywgLS1nZW5lcmFsLW51bWVyaWMtc29ydCAgY29tcGFyZSBhY2NvcmRp
bmcgdG8gZ2VuZXJhbCBudW1lcmljYWwgdmFsdWVcblwKKyAgLWksIC0taWdub3JlLWNhc2UgICBp
Z25vcmUgZGlmZmVyZW5jZXMgaW4gY2FzZSB3aGVuIGNvbXBhcmluZyBmaWVsZHNcblwKKyAgLW4s
IC0tbnVtZXJpYy1zb3J0ICBjb21wYXJlIGFjY29yZGluZyB0byBzdHJpbmcgbnVtZXJpY2FsIHZh
bHVlXG5cCiAgIC1qIEZJRUxEICAgICAgICAgIGVxdWl2YWxlbnQgdG8gJy0xIEZJRUxEIC0yIEZJ
RUxEJ1xuXAogICAtbyBGT1JNQVQgICAgICAgICBvYmV5IEZPUk1BVCB3aGlsZSBjb25zdHJ1Y3Rp
bmcgb3V0cHV0IGxpbmVcblwKICAgLXQgQ0hBUiAgICAgICAgICAgdXNlIENIQVIgYXMgaW5wdXQg
YW5kIG91dHB1dCBmaWVsZCBzZXBhcmF0b3JcblwKQEAgLTMwMyw2ICszMzMsNzMgQEAgZnJlZWxp
bmUgKHN0cnVjdCBsaW5lICpsaW5lKQogICBsaW5lLT5idWYuYnVmZmVyID0gTlVMTDsKIH0KIAor
LyogQ29tcGFyZSBzdHJpbmdzIEEgYW5kIEIgYXMgbnVtYmVycyB3aXRob3V0IGV4cGxpY2l0bHkg
Y29udmVydGluZyB0aGVtIHRvCisgICBtYWNoaW5lIG51bWJlcnMuICBDb21wYXJhdGl2ZWx5IHNs
b3cgZm9yIHNob3J0IHN0cmluZ3MsIGJ1dCBhc3ltcHRvdGljYWxseQorICAgaGlkZW91c2x5IGZh
c3QuIENvcGllZCBmcm9tIHNvcnQuYy4gKi8KKworc3RhdGljIGludAorbnVtY29tcGFyZSAoY2hh
ciBjb25zdCAqYSwgY2hhciBjb25zdCAqYikKK3sKKyAgd2hpbGUgKGJsYW5rc1t0b191Y2hhciAo
KmEpXSkKKyAgICBhKys7CisgIHdoaWxlIChibGFua3NbdG9fdWNoYXIgKCpiKV0pCisgICAgYisr
OworCisgIHJldHVybiBzdHJudW1jbXAgKGEsIGIsIGRlY2ltYWxfcG9pbnQsIHRob3VzYW5kc19z
ZXApOworfQorCisvKiBXb3JrIGFyb3VuZCBhIHByb2JsZW0gd2hlcmVieSB0aGUgbG9uZyBkb3Vi
bGUgdmFsdWUgcmV0dXJuZWQgYnkgZ2xpYmMncworICAgc3RydG9sZCAoIk5hTiIsIC4uLikgY29u
dGFpbnMgdW5pbml0aWFsaXplZCBiaXRzOiBjbGVhciBhbGwgYnl0ZXMgb2YKKyAgIEEgYW5kIEIg
YmVmb3JlIGNhbGxpbmcgc3RydG9sZC4gIEZJWE1FOiByZW1vdmUgdGhpcyBmdW5jdGlvbiBvbmNl
CisgICBnbnVsaWIgZ3VhcmFudGVlcyB0aGF0IHN0cnRvbGQncyByZXN1bHQgaXMgYWx3YXlzIHdl
bGwgZGVmaW5lZC4KKyAgIENvcGllZCBmcm9tIHNvcnQuYy4gKi8KKworc3RhdGljIGludAorbmFu
X2NvbXBhcmUgKGNoYXIgY29uc3QgKnNhLCBjaGFyIGNvbnN0ICpzYikKK3sKKyAgbG9uZ19kb3Vi
bGUgYTsKKyAgbWVtc2V0ICgmYSwgMCwgc2l6ZW9mIGEpOworICBhID0gc3RydG9sZCAoc2EsIE5V
TEwpOworCisgIGxvbmdfZG91YmxlIGI7CisgIG1lbXNldCAoJmIsIDAsIHNpemVvZiBiKTsKKyAg
YiA9IHN0cnRvbGQgKHNiLCBOVUxMKTsKKworICByZXR1cm4gbWVtY21wICgmYSwgJmIsIHNpemVv
ZiBhKTsKK30KKworCisvKiBDb21wYXJlIGdlbmVyYWwgbnVtZXJpY2FsIHZhbHVlIGJ5IGNvbnZl
cnNpb24gdG8gbG9uZyBkb3VibGUuCisgKiBDb3BpZWQgZnJvbSBzb3J0LmMuICovCisKK3N0YXRp
YyBpbnQKK2dlbmVyYWxfbnVtY29tcGFyZSAoY2hhciBjb25zdCAqc2EsIGNoYXIgY29uc3QgKnNi
KQoreworICAvKiBGSVhNRTogbWF5YmUgYWRkIG9wdGlvbiB0byB0cnkgZXhwZW5zaXZlIEZQIGNv
bnZlcnNpb24KKyAgICAgb25seSBpZiBBIGFuZCBCIGNhbid0IGJlIGNvbXBhcmVkIG1vcmUgY2hl
YXBseS9hY2N1cmF0ZWx5LiAgKi8KKworICBjaGFyICplYTsKKyAgY2hhciAqZWI7CisgIGxvbmdf
ZG91YmxlIGEgPSBzdHJ0b2xkIChzYSwgJmVhKTsKKyAgbG9uZ19kb3VibGUgYiA9IHN0cnRvbGQg
KHNiLCAmZWIpOworCisgIC8qIFB1dCBjb252ZXJzaW9uIGVycm9ycyBhdCB0aGUgc3RhcnQgb2Yg
dGhlIGNvbGxhdGluZyBzZXF1ZW5jZS4gICovCisgIGlmIChzYSA9PSBlYSkKKyAgICByZXR1cm4g
c2IgPT0gZWIgPyAwIDogLTE7CisgIGlmIChzYiA9PSBlYikKKyAgICByZXR1cm4gMTsKKworICAv
KiBTb3J0IG51bWJlcnMgaW4gdGhlIHVzdWFsIHdheSwgd2hlcmUgLTAgPT0gKzAuICBQdXQgTmFO
cyBhZnRlcgorICAgICBjb252ZXJzaW9uIGVycm9ycyBidXQgYmVmb3JlIG51bWJlcnM7IHNvcnQg
dGhlbSBieSBpbnRlcm5hbAorICAgICBiaXQtcGF0dGVybiwgZm9yIGxhY2sgb2YgYSBtb3JlIHBv
cnRhYmxlIGFsdGVybmF0aXZlLiAgKi8KKyAgcmV0dXJuIChhIDwgYiA/IC0xCisgICAgICAgICAg
OiBhID4gYiA/IDEKKyAgICAgICAgICA6IGEgPT0gYiA/IDAKKyAgICAgICAgICA6IGIgPT0gYiA/
IC0xCisgICAgICAgICAgOiBhID09IGEgPyAxCisgICAgICAgICAgOiBuYW5fY29tcGFyZSAoc2Es
IHNiKSk7Cit9CisKIC8qIFJldHVybiA8MCBpZiB0aGUgam9pbiBmaWVsZCBpbiBMSU5FMSBjb21w
YXJlcyBsZXNzIHRoYW4gdGhlIG9uZSBpbiBMSU5FMjsKICAgID4wIGlmIGl0IGNvbXBhcmVzIGdy
ZWF0ZXI7IDAgaWYgaXQgY29tcGFyZXMgZXF1YWwuCiAgICBSZXBvcnQgYW4gZXJyb3IgYW5kIGV4
aXQgaWYgdGhlIGNvbXBhcmlzb24gZmFpbHMuCkBAIC0zMTgsNiArNDE1LDcgQEAga2V5Y21wIChz
dHJ1Y3QgbGluZSBjb25zdCAqbGluZTEsIHN0cnVjdCBsaW5lIGNvbnN0ICpsaW5lMiwKIAogICBz
aXplX3QgbGVuMTsKICAgc2l6ZV90IGxlbjI7CQkvKiBMZW5ndGggb2YgZmllbGRzIHRvIGNvbXBh
cmUuICAqLworICBsb25nIGRvdWJsZSB4MSwgeDI7CiAgIGludCBkaWZmOwogCiAgIGlmIChqZl8x
IDwgbGluZTEtPm5maWVsZHMpCkBAIC0zNDcsNyArNDQ1LDE5IEBAIGtleWNtcCAoc3RydWN0IGxp
bmUgY29uc3QgKmxpbmUxLCBzdHJ1Y3QgbGluZSBjb25zdCAqbGluZTIsCiAgIGlmIChsZW4yID09
IDApCiAgICAgcmV0dXJuIDE7CiAKLSAgaWYgKGlnbm9yZV9jYXNlKQorICBpZiAobnVtZXJpY19z
b3J0KQorICAgIHsKKyAgICAgIGNoYXIgZW5kMSA9IGJlZzFbbGVuMV07CisgICAgICBjaGFyIGVu
ZDIgPSBiZWcyW2xlbjJdOworICAgICAgYmVnMVtsZW4xXSA9ICdcMCc7IGJlZzJbbGVuMl0gPSAn
XDAnOworICAgICAgZGlmZiA9IG51bWNvbXBhcmUgKGJlZzEsIGJlZzIpOworICAgICAgYmVnMVts
ZW4xXSA9IGVuZDE7IGJlZzJbbGVuMl0gPSBlbmQyOworICAgIH0KKyAgZWxzZSBpZiAoZ2VuZXJh
bF9udW1lcmljX3NvcnQpCisgICAgeworICAgICAgICBkaWZmID0gZ2VuZXJhbF9udW1jb21wYXJl
IChiZWcxLCBiZWcyKTsKKyAgICB9CisgIGVsc2UgaWYgKGlnbm9yZV9jYXNlKQogICAgIHsKICAg
ICAgIC8qIEZJWE1FOiBpZ25vcmVfY2FzZSBkb2VzIG5vdCB3b3JrIHdpdGggTkxTIChpbiBwYXJ0
aWN1bGFyLAogICAgICAgICAgd2l0aCBtdWx0aWJ5dGUgY2hhcnMpLiAgKi8KQEAgLTM2MCw3ICs0
NzAsNyBAQCBrZXljbXAgKHN0cnVjdCBsaW5lIGNvbnN0ICpsaW5lMSwgc3RydWN0IGxpbmUgY29u
c3QgKmxpbmUyLAogICAgICAgZGlmZiA9IG1lbWNtcCAoYmVnMSwgYmVnMiwgTUlOIChsZW4xLCBs
ZW4yKSk7CiAgICAgfQogCi0gIGlmIChkaWZmKQorICBpZiAoZGlmZiB8fCBudW1lcmljX3NvcnQg
fHwgZ2VuZXJhbF9udW1lcmljX3NvcnQpCiAgICAgcmV0dXJuIGRpZmY7CiAgIHJldHVybiBsZW4x
IDwgbGVuMiA/IC0xIDogbGVuMSAhPSBsZW4yOwogfQpAQCAtNDEyLDYgKzUyMiwxOSBAQCBjaGVj
a19vcmRlciAoY29uc3Qgc3RydWN0IGxpbmUgKnByZXYsCiAgICAgfQogfQogCisvKiBJbml0aWFs
aXplIHRoZSBjaGFyYWN0ZXIgY2xhc3MgdGFibGVzLiAqLworCitzdGF0aWMgdm9pZAoraW5pdHRh
YmxlcyAodm9pZCkKK3sKKyAgc2l6ZV90IGk7CisKKyAgZm9yIChpID0gMDsgaSA8IFVDSEFSX0xJ
TTsgKytpKQorICAgIHsKKyAgICAgIGJsYW5rc1tpXSA9ICEhIGlzYmxhbmsgKGkpOworICAgIH0K
K30KKwogc3RhdGljIGlubGluZSB2b2lkCiByZXNldF9saW5lIChzdHJ1Y3QgbGluZSAqbGluZSkK
IHsKQEAgLTEwMDksNiArMTEzMiwyNCBAQCBtYWluIChpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAg
IHRleHRkb21haW4gKFBBQ0tBR0UpOwogICBoYXJkX0xDX0NPTExBVEUgPSBoYXJkX2xvY2FsZSAo
TENfQ09MTEFURSk7CiAKKyAgLyogR2V0IGxvY2FsZSdzIHJlcHJlc2VudGF0aW9uIG9mIHRoZSBk
ZWNpbWFsIHBvaW50LiAgKi8KKyAgeworICAgIHN0cnVjdCBsY29udiBjb25zdCAqbG9jYWxlID0g
bG9jYWxlY29udiAoKTsKKworICAgIC8qIElmIHRoZSBsb2NhbGUgZG9lc24ndCBkZWZpbmUgYSBk
ZWNpbWFsIHBvaW50LCBvciBpZiB0aGUgZGVjaW1hbAorICAgICAgIHBvaW50IGlzIG11bHRpYnl0
ZSwgdXNlIHRoZSBDIGxvY2FsZSdzIGRlY2ltYWwgcG9pbnQuICBGSVhNRToKKyAgICAgICBhZGQg
c3VwcG9ydCBmb3IgbXVsdGlieXRlIGRlY2ltYWwgcG9pbnRzLiAgKi8KKyAgICBkZWNpbWFsX3Bv
aW50ID0gdG9fdWNoYXIgKGxvY2FsZS0+ZGVjaW1hbF9wb2ludFswXSk7CisgICAgaWYgKCEgZGVj
aW1hbF9wb2ludCB8fCBsb2NhbGUtPmRlY2ltYWxfcG9pbnRbMV0pCisgICAgICBkZWNpbWFsX3Bv
aW50ID0gJy4nOworCisgICAgLyogRklYTUU6IGFkZCBzdXBwb3J0IGZvciBtdWx0aWJ5dGUgdGhv
dXNhbmRzIHNlcGFyYXRvcnMuICAqLworICAgIHRob3VzYW5kc19zZXAgPSB0b191Y2hhciAoKmxv
Y2FsZS0+dGhvdXNhbmRzX3NlcCk7CisgICAgaWYgKCEgdGhvdXNhbmRzX3NlcCB8fCBsb2NhbGUt
PnRob3VzYW5kc19zZXBbMV0pCisgICAgICB0aG91c2FuZHNfc2VwID0gLTE7CisgIH0KKyAgaW5p
dHRhYmxlcyAoKTsKKwogICBhdGV4aXQgKGNsb3NlX3N0ZG91dCk7CiAgIGF0ZXhpdCAoZnJlZV9z
cGFyZWxpbmUpOwogCkBAIC0xMDE3LDcgKzExNTgsNyBAQCBtYWluIChpbnQgYXJnYywgY2hhciAq
KmFyZ3YpCiAgIGlzc3VlZF9kaXNvcmRlcl93YXJuaW5nWzBdID0gaXNzdWVkX2Rpc29yZGVyX3dh
cm5pbmdbMV0gPSBmYWxzZTsKICAgY2hlY2tfaW5wdXRfb3JkZXIgPSBDSEVDS19PUkRFUl9ERUZB
VUxUOwogCi0gIHdoaWxlICgob3B0YyA9IGdldG9wdF9sb25nIChhcmdjLCBhcmd2LCAiLWE6ZTpp
MToyOmo6bzp0OnY6IiwKKyAgd2hpbGUgKChvcHRjID0gZ2V0b3B0X2xvbmcgKGFyZ2MsIGFyZ3Ys
ICItYTplOmdpbjE6MjpqOm86dDp2OiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBs
b25nb3B0cywgTlVMTCkpCiAgICAgICAgICAhPSAtMSkKICAgICB7CkBAIC0xMDUwLDEwICsxMTkx
LDE4IEBAIG1haW4gKGludCBhcmdjLCBjaGFyICoqYXJndikKICAgICAgICAgICBlbXB0eV9maWxs
ZXIgPSBvcHRhcmc7CiAgICAgICAgICAgYnJlYWs7CiAKKyAgICAgICAgY2FzZSAnZyc6CisgICAg
ICAgICAgZ2VuZXJhbF9udW1lcmljX3NvcnQgPSB0cnVlOworICAgICAgICAgIGJyZWFrOworCiAg
ICAgICAgIGNhc2UgJ2knOgogICAgICAgICAgIGlnbm9yZV9jYXNlID0gdHJ1ZTsKICAgICAgICAg
ICBicmVhazsKIAorICAgICAgICBjYXNlICduJzoKKyAgICAgICAgICBudW1lcmljX3NvcnQgPSB0
cnVlOworICAgICAgICAgIGJyZWFrOworCiAgICAgICAgIGNhc2UgJzEnOgogICAgICAgICAgIHNl
dF9qb2luX2ZpZWxkICgmam9pbl9maWVsZF8xLCBzdHJpbmdfdG9fam9pbl9maWVsZCAob3B0YXJn
KSk7CiAgICAgICAgICAgYnJlYWs7CmRpZmYgLS1naXQgYS90ZXN0cy9taXNjL2pvaW4gYi90ZXN0
cy9taXNjL2pvaW4KaW5kZXggYTNmZDFhOC4uMmRjZTk0ZSAxMDA3NTUKLS0tIGEvdGVzdHMvbWlz
Yy9qb2luCisrKyBiL3Rlc3RzL21pc2Mvam9pbgpAQCAtMTQ3LDYgKzE0NywxNCBAQCBteSBAdHYg
PSAoCiAgWyJhLDEsLDJcbmIsMSwyXG4iLCAiYSwzLDRcbmIsMyw0XG4iXSwKICAiYSwxLCwyLDMs
NFxuYiwxLDIsLDMsNFxuIl0sCiAKKyMgSm9pbiBvbiBudW1lcmljYWxseSBzb3J0ZWQgZmllbGQu
CitbJ251bWVyaWMtMScsICctbicsIFsiNyBzXG44IGVcbjEwIHRcbjQwMDAgZlxuIiwgIjcgU1xu
OSBOXG40MDAwIEZcbiJdLAorICAgICI3IHMgU1xuNDAwMCBmIEZcbiIsIDBdLAorWydudW1lcmlj
LTInLCAnJywgICBbIjcgc1xuOCBlXG4xMCB0XG4iLCAiNyBTXG45IE5cbjEwIFRcbiJdLAorICAg
ICI3IHMgU1xuIiwgMSwgIiRwcm9nOiBudW1lcmljLTIuMjozOiBpcyBub3Qgc29ydGVkOiAxMCBU
XG4iXSwKK1snbnVtZXJpYy0zJywgJy1nJywgICBbIjcgc1xuNWUyIGZcbjgwMCBlIiwgIjcwZS0x
IFNcbjUwMC4wIEZcbiJdLAorICAgICI3IHMgU1xuNWUyIGYgRlxuIiwgMF0sCisKICMgRnJvbSBU
aW0gU21pdGhlcnM6IGZpeGVkIGluIDEuMjJsCiBbJ3RyYWlsaW5nLXNwJywgJy10OiAtMSAxIC0y
IDEnLCBbImE6eCBcbiIsICJhOnkgXG4iXSwgImE6eCA6eSBcbiIsIDBdLAogCi0tIAoxLjcuOS40
Cgo=
--e89a8f83a645acfaf804bbba26a5--




Information forwarded to bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 9 Jun 2010 17:07:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 09 13:07:07 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OMOk7-0001bR-LO
	for submit <at> debbugs.gnu.org; Wed, 09 Jun 2010 13:07:07 -0400
Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <eggert@HIDDEN>) id 1OMOk6-0001b5-PK
	for submit <at> debbugs.gnu.org; Wed, 09 Jun 2010 13:07:07 -0400
Received: from lists.gnu.org ([199.232.76.165]:59982)
	by monty-python.gnu.org with esmtps
	(TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60)
	(envelope-from <eggert@HIDDEN>) id 1OMOk2-0003TS-1j
	for submit <at> debbugs.gnu.org; Wed, 09 Jun 2010 13:07:02 -0400
Received: from [140.186.70.92] (port=38018 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1OMOk0-00042I-Up
	for bug-coreutils@HIDDEN; Wed, 09 Jun 2010 13:07:01 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD
	autolearn=unavailable version=3.3.1
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <eggert@HIDDEN>) id 1OMOjz-0003YU-Qj
	for bug-coreutils@HIDDEN; Wed, 09 Jun 2010 13:07:00 -0400
Received: from kiwi.cs.ucla.edu ([131.179.128.19]:45993)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <eggert@HIDDEN>) id 1OMOjz-0003Y5-Ew
	for bug-coreutils@HIDDEN; Wed, 09 Jun 2010 13:06:59 -0400
Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200])
	by kiwi.cs.ucla.edu (8.13.8+Sun/8.13.8/UCLACS-6.0) with ESMTP id
	o59H6u5W008034
	for <bug-coreutils@HIDDEN>; Wed, 9 Jun 2010 10:06:57 -0700 (PDT)
Message-ID: <4C0FCA30.90203@HIDDEN>
Date: Wed, 09 Jun 2010 10:06:56 -0700
From: Paul Eggert <eggert@HIDDEN>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.9) Gecko/20100423 Thunderbird/3.0.4
MIME-Version: 1.0
To: bug-coreutils@HIDDEN
Subject: Re: bug#6366: join can't join on numeric fields
References: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>	<4C0D79C1.5080206@HIDDEN>	<AANLkTik_l7BpI2_RyOk1nJ5ZFzaqUIE0CTDQL8mcKb25@HIDDEN>
	<87ocfkr8xk.fsf@HIDDEN>
In-Reply-To: <87ocfkr8xk.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta)
X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6,
	seldom 2.4 (older, 4)
X-Spam-Score: -4.4 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -4.5 (----)

On 06/08/2010 11:56 PM, Jim Meyering wrote:

> There may be a fly in the ointment.
> 
> When comparing floating point numbers how would join measure equality?

The point is that "join" should be compatible with "sort".
Any option that "sort" has to compare fields,
is an option that "join" should also have.
The same code should be used for both "join" and "sort",
to do comparison.  So, if "sort" has an option to do
IEEE-754 comparison in a certain way, "join" should
have the same option.

Arguably "uniq" should have the same set of options,
when checking whether two lines are equal, but I'd
say that's lower priority.




Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 9 Jun 2010 07:33:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 09 03:33:12 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OMFmh-0004fY-Td
	for submit <at> debbugs.gnu.org; Wed, 09 Jun 2010 03:33:12 -0400
Received: from mail-pw0-f44.google.com ([209.85.160.44])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <alexshinn@HIDDEN>) id 1OMFmf-0004fT-V1
	for 6366 <at> debbugs.gnu.org; Wed, 09 Jun 2010 03:33:10 -0400
Received: by pwi8 with SMTP id 8so2433460pwi.3
	for <6366 <at> debbugs.gnu.org>; Wed, 09 Jun 2010 00:33:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:cc:content-type;
	bh=HeqMZn76cP84r3wcgLKxa1THWcx1XnTKPqP+xV7zoLc=;
	b=TMzaVNmrhDSVPVUTQEssKtIybZZBQqOrsmkKKfkvuC/ib5WB3NyzSI2d9SKG2LLSw0
	CT5teMJf+o1oxYwf+n77DOTwmlmVLW+hbK297B30b2dSufGa/VWqVMK3FgoF1sYzpyQ4
	BDt0vQCLw0qp8/VhAbg00fb+J2d/DgGqd+tro=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	b=ME9/8Dc2TSczCIqIxSstKRmQNyEHGobWHSV/Ace6quZGGKFmYPRguiwTMyY//UfPIY
	XRPfSX+PqIYlFicmHxIngCDOxflSKc9NMoenPLvBvud6sdoWNfllUAkK8rWU9o8fBGfo
	3zxmsLDaReD7RsYbv7QC5FTQUi8rtr2ltJCc0=
MIME-Version: 1.0
Received: by 10.142.67.2 with SMTP id p2mr2600798wfa.288.1276068786149; Wed, 
	09 Jun 2010 00:33:06 -0700 (PDT)
Received: by 10.142.43.6 with HTTP; Wed, 9 Jun 2010 00:33:06 -0700 (PDT)
In-Reply-To: <87ocfkr8xk.fsf@HIDDEN>
References: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
	<4C0D79C1.5080206@HIDDEN>
	<AANLkTik_l7BpI2_RyOk1nJ5ZFzaqUIE0CTDQL8mcKb25@HIDDEN>
	<87ocfkr8xk.fsf@HIDDEN>
Date: Wed, 9 Jun 2010 16:33:06 +0900
Message-ID: <AANLkTimMR25wi5PdpEcCaMUgHLJEpMApYZ8oP9j9VffY@HIDDEN>
Subject: Re: bug#6366: join can't join on numeric fields
From: Alex Shinn <alexshinn@HIDDEN>
To: Jim Meyering <jim@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 6366
Cc: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@HIDDEN>, 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.9 (--)

On Wed, Jun 9, 2010 at 3:56 PM, Jim Meyering <jim@HIDDEN> wrote:
>
> When comparing floating point numbers how would join measure equality?
> Should it consider 1.000000000000001e2 to be equal to 100.0 ?
> What if the maximum precision available does not
> allow us to distinguish those two values?

Indeed, that's why you generally don't use floating
point numbers as database keys.

We could either restrict the numbers to integers,
or continue to allow floats with a note to the effect
that float precision is machine-specific.  Personally
I have no need for floats so am happy with the former.

-- 
Alex




Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 9 Jun 2010 06:56:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 09 02:56:20 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OMFD2-0004OJ-9z
	for submit <at> debbugs.gnu.org; Wed, 09 Jun 2010 02:56:20 -0400
Received: from smtp1-g21.free.fr ([212.27.42.1])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <meyering@HIDDEN>) id 1OMFCz-0004OD-79
	for 6366 <at> debbugs.gnu.org; Wed, 09 Jun 2010 02:56:18 -0400
Received: from mx.meyering.net (mx.meyering.net [82.230.74.64])
	by smtp1-g21.free.fr (Postfix) with ESMTP id 187D294012A
	for <6366 <at> debbugs.gnu.org>; Wed,  9 Jun 2010 08:56:08 +0200 (CEST)
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id DDD24CD5; Wed,  9 Jun 2010 08:56:07 +0200 (CEST)
From: Jim Meyering <jim@HIDDEN>
To: Alex Shinn <alexshinn@HIDDEN>
Subject: Re: bug#6366: join can't join on numeric fields
In-Reply-To: <AANLkTik_l7BpI2_RyOk1nJ5ZFzaqUIE0CTDQL8mcKb25@HIDDEN>
	(Alex Shinn's message of "Wed, 9 Jun 2010 10:47:05 +0900")
References: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
	<4C0D79C1.5080206@HIDDEN>
	<AANLkTik_l7BpI2_RyOk1nJ5ZFzaqUIE0CTDQL8mcKb25@HIDDEN>
Date: Wed, 09 Jun 2010 08:56:07 +0200
Message-ID: <87ocfkr8xk.fsf@HIDDEN>
Lines: 51
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -3.3 (---)
X-Debbugs-Envelope-To: 6366
Cc: =?utf-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>, 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.3 (---)

Alex Shinn wrote:

> 2010/6/8 P=C3=A1draig Brady <P@HIDDEN>:
>> On 07/06/10 06:19, Alex Shinn wrote:
>>>
>>> Ideally join should be able to handle files sorted in any order
>>> that sort provides, but as a bare minimum it should at least
>>> be able to join files sorted on numeric fields.
>>
>> Well if there were no aliases in the numbers, you could always
>> sort the output numerically after the join if it was important.
>
> By first sorting lexicographically, you mean?
> In the use case I had, the data was already sorted
> numerically.  So whenever I want to join two files,
> currently I have to do:
>
>   sort file1 > file1.tmp
>   sort file2 > file2.tmp
>   join file1.tmp file2.tmp | sort -n > out
>   rm -f file1.tmp file2.tmp
>
> instead of just
>
>   join -n file1 file2 > out
>
> In the small tools philosophy you want to avoid adding
> redundancy, but in this case join isn't doing the same
> thing as sort, it's just working with it better.  Not to mention
> the fact that sort is an expensive operation to have to
> perform multiple times, not just an extra O(n) filter
> to throw in the middle of a pipeline.
>
>> However if you wanted to join "01" and "1" then your patch is required.
>> Are numeric aliases common enough to warrant this? I think so.
>
> Leading zeros may not be so common, but don't forget
> "1.0" and "1" or "1e2" and "100" and "100.0", etc.
>
>> I'd use -g, --general-numeric to correspond with `sort`.
>
> Yes, that's probably better.

There may be a fly in the ointment.

When comparing floating point numbers how would join measure equality?
Should it consider 1.000000000000001e2 to be equal to 100.0 ?
What if the maximum precision available does not
allow us to distinguish those two values?

What about -0 and 0? (with IEEE 754, they'll compare equal)




Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 9 Jun 2010 01:47:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 08 21:47:12 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OMANs-0002NR-Fo
	for submit <at> debbugs.gnu.org; Tue, 08 Jun 2010 21:47:12 -0400
Received: from mail-pz0-f182.google.com ([209.85.222.182])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <alexshinn@HIDDEN>) id 1OMANq-0002NM-0b
	for 6366 <at> debbugs.gnu.org; Tue, 08 Jun 2010 21:47:11 -0400
Received: by pzk12 with SMTP id 12so3578724pzk.9
	for <6366 <at> debbugs.gnu.org>; Tue, 08 Jun 2010 18:47:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:cc:content-type
	:content-transfer-encoding;
	bh=4ibj+mw/hkom9vzCxxj2TPsno6i7tD7oaTz8B/gSiDA=;
	b=Vd2PRDDRhPHRgE02MSsBTt6L/nfp/wCsIwJavXNlLlVxo3QgH2DKz9CLk1MYsZFbpa
	msjDXBlSqH6XvwL9WWzx0IYVtiNaFy8uFbcg/YclA+onjEK6dctmmhByd/7Tg+i/e4/S
	q42UC28gNwLrkxbq+DRpN8ZCUskBdTKLdaiY4=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=DkU/TI+Ot/Cwlitn4LP0din5RGJef38tdht6sPYg2ZccafsRXL0EBZWqeNkGAstwTG
	GakGn62OA8EBKPFH1I3ociLK3RRie6T/7dfFUi3AeYW+/BsgSAtmrS0juz8c9STPT4Xi
	osYHkes/t4vLYZnEmtEWv2jRFJ96XPKSJ/DXU=
MIME-Version: 1.0
Received: by 10.142.202.15 with SMTP id z15mr1777454wff.281.1276048025600; 
	Tue, 08 Jun 2010 18:47:05 -0700 (PDT)
Received: by 10.142.43.6 with HTTP; Tue, 8 Jun 2010 18:47:05 -0700 (PDT)
In-Reply-To: <4C0D79C1.5080206@HIDDEN>
References: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
	<4C0D79C1.5080206@HIDDEN>
Date: Wed, 9 Jun 2010 10:47:05 +0900
Message-ID: <AANLkTik_l7BpI2_RyOk1nJ5ZFzaqUIE0CTDQL8mcKb25@HIDDEN>
Subject: Re: bug#6366: join can't join on numeric fields
From: Alex Shinn <alexshinn@HIDDEN>
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 6366
Cc: 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.0 (---)

2010/6/8 P=E1draig Brady <P@HIDDEN>:
> On 07/06/10 06:19, Alex Shinn wrote:
>>
>> Ideally join should be able to handle files sorted in any order
>> that sort provides, but as a bare minimum it should at least
>> be able to join files sorted on numeric fields.
>
> Well if there were no aliases in the numbers, you could always
> sort the output numerically after the join if it was important.

By first sorting lexicographically, you mean?
In the use case I had, the data was already sorted
numerically.  So whenever I want to join two files,
currently I have to do:

  sort file1 > file1.tmp
  sort file2 > file2.tmp
  join file1.tmp file2.tmp | sort -n > out
  rm -f file1.tmp file2.tmp

instead of just

  join -n file1 file2 > out

In the small tools philosophy you want to avoid adding
redundancy, but in this case join isn't doing the same
thing as sort, it's just working with it better.  Not to mention
the fact that sort is an expensive operation to have to
perform multiple times, not just an extra O(n) filter
to throw in the middle of a pipeline.

> However if you wanted to join "01" and "1" then your patch is required.
> Are numeric aliases common enough to warrant this? I think so.

Leading zeros may not be so common, but don't forget
"1.0" and "1" or "1e2" and "100" and "100.0", etc.

> I'd use -g, --general-numeric to correspond with `sort`.

Yes, that's probably better.

--=20
Alex




Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.

Message received at 6366 <at> debbugs.gnu.org:


Received: (at 6366) by debbugs.gnu.org; 7 Jun 2010 22:59:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 07 18:59:56 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OLlIR-0004Tt-Rs
	for submit <at> debbugs.gnu.org; Mon, 07 Jun 2010 18:59:56 -0400
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98])
	by debbugs.gnu.org with smtp (Exim 4.69)
	(envelope-from <P@HIDDEN>) id 1OLlIP-0004Tm-P8
	for 6366 <at> debbugs.gnu.org; Mon, 07 Jun 2010 18:59:54 -0400
Received: (qmail 17603 invoked from network); 7 Jun 2010 22:59:49 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 7 Jun 2010 22:59:49 -0000
Message-ID: <4C0D79C1.5080206@HIDDEN>
Date: Mon, 07 Jun 2010 23:59:13 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@HIDDEN>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Alex Shinn <alexshinn@HIDDEN>
Subject: Re: bug#6366: join can't join on numeric fields
References: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
In-Reply-To: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Spam-Score: -1.5 (-)
X-Debbugs-Envelope-To: 6366
Cc: 6366 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.8 (--)

On 07/06/10 06:19, Alex Shinn wrote:
> Hi,
> 
> Ideally join should be able to handle files sorted in any order
> that sort provides, but as a bare minimum it should at least
> be able to join files sorted on numeric fields.

Well if there were no aliases in the numbers, you could always
sort the output numerically after the join if it was important.
However if you wanted to join "01" and "1" then your patch is required.
Are numeric aliases common enough to warrant this? I think so.

> The attached simple patch provides -n, --numeric-sort
> options to this effect.

I'd use -g, --general-numeric to correspond with `sort`.

cheers,
Pdraig.




Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.
Added tag(s) patch. Request was from bob@HIDDEN (Bob Proulx) to control <at> debbugs.gnu.org. Full text available.
Severity set to 'wishlist' from 'normal' Request was from bob@HIDDEN (Bob Proulx) to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 7 Jun 2010 05:23:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 07 01:23:38 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OLUoE-0003LN-6J
	for submit <at> debbugs.gnu.org; Mon, 07 Jun 2010 01:23:38 -0400
Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <alexshinn@HIDDEN>) id 1OLUkL-0003JK-5s
	for submit <at> debbugs.gnu.org; Mon, 07 Jun 2010 01:19:37 -0400
Received: from lists.gnu.org ([199.232.76.165]:52868)
	by monty-python.gnu.org with esmtps
	(TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60)
	(envelope-from <alexshinn@HIDDEN>) id 1OLUkH-0003YM-PI
	for submit <at> debbugs.gnu.org; Mon, 07 Jun 2010 01:19:33 -0400
Received: from [140.186.70.92] (port=60075 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1OLUkG-0004wM-AJ
	for bug-coreutils@HIDDEN; Mon, 07 Jun 2010 01:19:33 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM,
	T_DKIM_INVALID,
	T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <alexshinn@HIDDEN>) id 1OLUkB-0005XG-Ay
	for bug-coreutils@HIDDEN; Mon, 07 Jun 2010 01:19:32 -0400
Received: from mail-bw0-f41.google.com ([209.85.214.41]:36831)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <alexshinn@HIDDEN>) id 1OLUkB-0005X3-22
	for bug-coreutils@HIDDEN; Mon, 07 Jun 2010 01:19:27 -0400
Received: by bwz1 with SMTP id 1so1082627bwz.0
	for <bug-coreutils@HIDDEN>; Sun, 06 Jun 2010 22:19:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:date:message-id
	:subject:from:to:content-type;
	bh=x3p3mXnBrhWx6mu73X55+qHDF6CvOqlKiF93TK+/0q8=;
	b=m/41qm0a9loKDuONPeeFQKlAqWfMlHBpZlPF2bKFYTh4xf4EvbwK2ZiPhz2VVKI2e/
	o0KuQzbS0wYMRgSDCKfI13mN54bR+LJLi+Mh18nyDqtWU9hPuwH5pBoGhQ2YF1KYtzD2
	MOebKY9nEysB3cOzuEv1LKmaviIKbgTn6NTec=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	b=nhDZb4ZNn4vBObc1pFlxr1Og7ddbDfylycVUiZtQNqIUjL7NWZQEqvhgIHw8tnEIK1
	X4DCSbUSGQlXwGzZ/958SE3w9Zd9h11i0uRTiHNZYiAx7NLHAsBXNtPm20pKa51Tb5Nd
	nIK4CrI4hS0A7T+76VQV2WqDkIEY0ON5Q/OvA=
MIME-Version: 1.0
Received: by 10.204.79.197 with SMTP id q5mr6365826bkk.123.1275887964378; Sun, 
	06 Jun 2010 22:19:24 -0700 (PDT)
Received: by 10.204.129.3 with HTTP; Sun, 6 Jun 2010 22:19:24 -0700 (PDT)
Date: Mon, 7 Jun 2010 14:19:24 +0900
Message-ID: <AANLkTikFbKLKbHb8li_oqyRWLCYLAxFFhQwOuT1ll5RU@HIDDEN>
Subject: join can't join on numeric fields
From: Alex Shinn <alexshinn@HIDDEN>
To: bug-coreutils@HIDDEN
Content-Type: multipart/mixed; boundary=0016e6dd9902e3d8d2048869d036
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6,
	seldom 2.4 (older, 4)
X-Spam-Score: -3.3 (---)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Mon, 07 Jun 2010 01:23:37 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -4.6 (----)

--0016e6dd9902e3d8d2048869d036
Content-Type: text/plain; charset=ISO-8859-1

Hi,

Ideally join should be able to handle files sorted in any order
that sort provides, but as a bare minimum it should at least
be able to join files sorted on numeric fields.

The attached simple patch provides -n, --numeric-sort
options to this effect.

-- 
Alex

--0016e6dd9902e3d8d2048869d036
Content-Type: text/x-patch; charset=US-ASCII; name="join.c.diff"
Content-Disposition: attachment; filename="join.c.diff"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_ga4unz9c0

LS0tIHNyYy9qb2luLmN+CTIwMTAtMDQtMjEgMDQ6NTI6MDQuMDAwMDAwMDAwICswOTAwCisrKyBz
cmMvam9pbi5jCTIwMTAtMDUtMjcgMTU6MDQ6NDAuODk4NjUxNzMyICswOTAwCkBAIC0xNDUsNiAr
MTQ1LDcgQEAKIHN0YXRpYyBzdHJ1Y3Qgb3B0aW9uIGNvbnN0IGxvbmdvcHRzW10gPQogewogICB7
Imlnbm9yZS1jYXNlIiwgbm9fYXJndW1lbnQsIE5VTEwsICdpJ30sCisgIHsibnVtZXJpYy1zb3J0
Iiwgbm9fYXJndW1lbnQsIE5VTEwsICduJ30sCiAgIHsiY2hlY2stb3JkZXIiLCBub19hcmd1bWVu
dCwgTlVMTCwgQ0hFQ0tfT1JERVJfT1BUSU9OfSwKICAgeyJub2NoZWNrLW9yZGVyIiwgbm9fYXJn
dW1lbnQsIE5VTEwsIE5PQ0hFQ0tfT1JERVJfT1BUSU9OfSwKICAgeyJoZWFkZXIiLCBub19hcmd1
bWVudCwgTlVMTCwgSEVBREVSX0xJTkVfT1BUSU9OfSwKQEAgLTE1OSw2ICsxNjAsOSBAQAogLyog
SWYgbm9uemVybywgaWdub3JlIGNhc2Ugd2hlbiBjb21wYXJpbmcgam9pbiBmaWVsZHMuICAqLwog
c3RhdGljIGJvb2wgaWdub3JlX2Nhc2U7CiAKKy8qIElmIG5vbnplcm8sIHRyZWF0IGtleXMgYXMg
bnVtZXJpYyB2YWx1ZXMuICAqLworc3RhdGljIGJvb2wgbnVtZXJpY19zb3J0OworCiAvKiBJZiBu
b256ZXJvLCB0cmVhdCB0aGUgZmlyc3QgbGluZSBvZiBlYWNoIGZpbGUgYXMgY29sdW1uIGhlYWRl
cnMgLQogICAgam9pbiB0aGVtIHdpdGhvdXQgY2hlY2tpbmcgZm9yIG9yZGVyaW5nICovCiBzdGF0
aWMgYm9vbCBqb2luX2hlYWRlcl9saW5lczsKQEAgLTE4Niw2ICsxOTAsNyBAQAogIiksIHN0ZG91
dCk7CiAgICAgICBmcHV0cyAoXygiXAogICAtaSwgLS1pZ25vcmUtY2FzZSAgaWdub3JlIGRpZmZl
cmVuY2VzIGluIGNhc2Ugd2hlbiBjb21wYXJpbmcgZmllbGRzXG5cCisgIC1uLCAtLW51bWVyaWMt
c29ydCBjb21wYXJlIGFjY29yZGluZyB0byBzdHJpbmcgbnVtZXJpY2FsIGNhbHVlXG5cCiAgIC1q
IEZJRUxEICAgICAgICAgIGVxdWl2YWxlbnQgdG8gYC0xIEZJRUxEIC0yIEZJRUxEJ1xuXAogICAt
byBGT1JNQVQgICAgICAgICBvYmV5IEZPUk1BVCB3aGlsZSBjb25zdHJ1Y3Rpbmcgb3V0cHV0IGxp
bmVcblwKICAgLXQgQ0hBUiAgICAgICAgICAgdXNlIENIQVIgYXMgaW5wdXQgYW5kIG91dHB1dCBm
aWVsZCBzZXBhcmF0b3JcblwKQEAgLTMwNCw2ICszMDksNyBAQAogCiAgIHNpemVfdCBsZW4xOwog
ICBzaXplX3QgbGVuMjsJCS8qIExlbmd0aCBvZiBmaWVsZHMgdG8gY29tcGFyZS4gICovCisgIGxv
bmcgZG91YmxlIHgxLCB4MjsKICAgaW50IGRpZmY7CiAKICAgaWYgKGpmXzEgPCBsaW5lMS0+bmZp
ZWxkcykKQEAgLTMzMyw3ICszMzksMTMgQEAKICAgaWYgKGxlbjIgPT0gMCkKICAgICByZXR1cm4g
MTsKIAotICBpZiAoaWdub3JlX2Nhc2UpCisgIGlmIChudW1lcmljX3NvcnQpCisgICAgeworICAg
ICAgeDEgPSBzdHJ0b2xkKGJlZzEsIE5VTEwpOworICAgICAgeDIgPSBzdHJ0b2xkKGJlZzIsIE5V
TEwpOworICAgICAgZGlmZiA9IHgxIC0geDI7CisgICAgfQorICBlbHNlIGlmIChpZ25vcmVfY2Fz
ZSkKICAgICB7CiAgICAgICAvKiBGSVhNRTogaWdub3JlX2Nhc2UgZG9lcyBub3Qgd29yayB3aXRo
IE5MUyAoaW4gcGFydGljdWxhciwKICAgICAgICAgIHdpdGggbXVsdGlieXRlIGNoYXJzKS4gICov
CkBAIC05NzEsNyArOTgzLDcgQEAKICAgaXNzdWVkX2Rpc29yZGVyX3dhcm5pbmdbMF0gPSBpc3N1
ZWRfZGlzb3JkZXJfd2FybmluZ1sxXSA9IGZhbHNlOwogICBjaGVja19pbnB1dF9vcmRlciA9IENI
RUNLX09SREVSX0RFRkFVTFQ7CiAKLSAgd2hpbGUgKChvcHRjID0gZ2V0b3B0X2xvbmcgKGFyZ2Ms
IGFyZ3YsICItYTplOmkxOjI6ajpvOnQ6djoiLAorICB3aGlsZSAoKG9wdGMgPSBnZXRvcHRfbG9u
ZyAoYXJnYywgYXJndiwgIi1hOmU6aW4xOjI6ajpvOnQ6djoiLAogICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgbG9uZ29wdHMsIE5VTEwpKQogICAgICAgICAgIT0gLTEpCiAgICAgewpAQCAt
MTAwOCw2ICsxMDIwLDEwIEBACiAgICAgICAgICAgaWdub3JlX2Nhc2UgPSB0cnVlOwogICAgICAg
ICAgIGJyZWFrOwogCisgICAgICAgIGNhc2UgJ24nOgorICAgICAgICAgIG51bWVyaWNfc29ydCA9
IHRydWU7CisgICAgICAgICAgYnJlYWs7CisKICAgICAgICAgY2FzZSAnMSc6CiAgICAgICAgICAg
c2V0X2pvaW5fZmllbGQgKCZqb2luX2ZpZWxkXzEsIHN0cmluZ190b19qb2luX2ZpZWxkIChvcHRh
cmcpKTsKICAgICAgICAgICBicmVhazsK
--0016e6dd9902e3d8d2048869d036--




Acknowledgement sent to Alex Shinn <alexshinn@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-coreutils@HIDDEN. Full text available.
Report forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6366; Package coreutils. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 9 Oct 2018 20:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.