X-Loop: help-debbugs@HIDDEN Subject: bug#16471: New entropy coding: faster than Huffman, compression rate like arithmetic Resent-From: =?UTF-8?Q?Jaros=C5=82aw?= Duda <dudaj@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-gzip@HIDDEN Resent-Date: Thu, 16 Jan 2014 23:52:01 +0000 Resent-Message-ID: <handler.16471.B.138991628513192 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 16471 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: 16471 <at> debbugs.gnu.org X-Debbugs-Original-To: bug-gzip@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.138991628513192 (code B ref -1); Thu, 16 Jan 2014 23:52:01 +0000 Received: (at submit) by debbugs.gnu.org; 16 Jan 2014 23:51:25 +0000 Received: from localhost ([127.0.0.1]:54196 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1W3wia-0003Qf-Ej for submit <at> debbugs.gnu.org; Thu, 16 Jan 2014 18:51:24 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48647) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from <dudaj@HIDDEN>) id 1W3wfB-0003GQ-Iy for submit <at> debbugs.gnu.org; Thu, 16 Jan 2014 18:47:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <dudaj@HIDDEN>) id 1W3wf6-0006hF-OK for submit <at> debbugs.gnu.org; Thu, 16 Jan 2014 18:47:53 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:58383) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <dudaj@HIDDEN>) id 1W3wf6-0006hB-Kv for submit <at> debbugs.gnu.org; Thu, 16 Jan 2014 18:47:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50339) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <dudaj@HIDDEN>) id 1W3wf2-0004dc-6w for bug-gzip@HIDDEN; Thu, 16 Jan 2014 18:47:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <dudaj@HIDDEN>) id 1W3wex-0006gJ-UF for bug-gzip@HIDDEN; Thu, 16 Jan 2014 18:47:44 -0500 Received: from mailhub129.itcs.purdue.edu ([128.210.5.129]:53790) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <dudaj@HIDDEN>) id 1W3wex-0006g5-OS for bug-gzip@HIDDEN; Thu, 16 Jan 2014 18:47:39 -0500 Received: from [10.184.172.48] (pal-nat184-172-048.itap.purdue.edu [10.184.172.48]) (authenticated bits=0) by mailhub129.itcs.purdue.edu (8.14.4/8.14.4/mta-auth.smtp.purdue.edu) with ESMTP id s0GNlcWW032218 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for <bug-gzip@HIDDEN>; Thu, 16 Jan 2014 18:47:38 -0500 Message-ID: <52D86F98.1030301@HIDDEN> Date: Thu, 16 Jan 2014 18:47:36 -0500 From: =?UTF-8?Q?Jaros=C5=82aw?= Duda <dudaj@HIDDEN> User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed X-PMX-Version: 6.0.2.2308539 X-PerlMx-Virus-Scanned: Yes Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mailhub129.itcs.purdue.edu id s0GNlcWW032218 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Mailman-Approved-At: Thu, 16 Jan 2014 18:51:22 -0500 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -5.0 (-----) Hallo, Since I haven't found gzip development mailing list, I am sending it here. There is a new approach to entropy coding: asymmetric numeral systems.=20 It has some connection with arithmetic coding, but is simpler: uses a=20 single natural number as the state, instead of two to represent the=20 range. Thanks of it, we can put the entire behavior for given large=20 alphabet probability distribution into a relatively small table: a few=20 kilobytes for 256 size alphabet. This way it turns out about 50% faster than Huffman decoding, still=20 providing accuracy (compression rate) like arithmetic coding. Here is some implementation: https://github.com/Cyan4973/FiniteStateEntro= py Just replacing Huffman with it, we get improvement of both speed and=20 compression rate, like recently in Zhuff of Yann Collet:=20 http://fastcompression.blogspot.fr/2013/12/zhuff-v09-or-first-fse-applica= tion.html I thought it could also improve gzip DEFLATE in the same way? Best, Jarek -- dr Jaros=C5=82aw Duda Center for Science of Information, Purdue University, USA http://www.soihub.org/people.php?id=3D484
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.503 (Entity 5.503) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: =?UTF-8?Q?Jaros=C5=82aw?= Duda <dudaj@HIDDEN> Subject: bug#16471: Acknowledgement (New entropy coding: faster than Huffman, compression rate like arithmetic) Message-ID: <handler.16471.B.138991628513192.ack <at> debbugs.gnu.org> References: <52D86F98.1030301@HIDDEN> X-Gnu-PR-Message: ack 16471 X-Gnu-PR-Package: gzip Reply-To: 16471 <at> debbugs.gnu.org Date: Thu, 16 Jan 2014 23:52:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-gzip@HIDDEN If you wish to submit further information on this problem, please send it to 16471 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 16471: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D16471 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
Received: (at control) by debbugs.gnu.org; 10 Nov 2014 18:39:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Nov 10 13:39:41 2014 Received: from localhost ([127.0.0.1]:56265 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1XntsL-0000ZK-Ki for submit <at> debbugs.gnu.org; Mon, 10 Nov 2014 13:39:41 -0500 Received: from smtp.cs.ucla.edu ([131.179.128.62]:60213) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from <eggert@HIDDEN>) id 1XntsJ-0000ZB-VI for control <at> debbugs.gnu.org; Mon, 10 Nov 2014 13:39:40 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp.cs.ucla.edu (Postfix) with ESMTP id 41002A60040 for <control <at> debbugs.gnu.org>; Mon, 10 Nov 2014 10:39:39 -0800 (PST) X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu Received: from smtp.cs.ucla.edu ([127.0.0.1]) by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G1K0liKed5Yl for <control <at> debbugs.gnu.org>; Mon, 10 Nov 2014 10:39:30 -0800 (PST) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 70436A60043 for <control <at> debbugs.gnu.org>; Mon, 10 Nov 2014 10:39:30 -0800 (PST) Message-ID: <54610662.6060706@HIDDEN> Date: Mon, 10 Nov 2014 10:39:30 -0800 From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: control <at> debbugs.gnu.org Subject: gzip bug report maintenance Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.9 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.9 (--) severity 18000 wishlist severity 17804 wishlist severity 16471 wishlist
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.