GNU bug report logs - #10338
23.3; Firefox includes a jar file which is not recognied as a zip file (omni.jar)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10338 in the body.
You can then email your comments to 10338 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Brooks, Daniel" <Daniel.Brooks <at> ask.com>
To: "bug-gnu-emacs <at> gnu.org" <bug-gnu-emacs <at> gnu.org>
Subject: 23.3; Firefox includes a jar file which is not recognied as a zip
	file (omni.jar)
Date: Tue, 20 Dec 2011 21:51:18 -0500

[Message part 1 (text/plain, inline)]

--=-=-=



        They've optimized for read performance by rearranging the sections of

the file (and the files inside the archive). As a result the magic is

slightly different and Emacs doesn't recognize the file, even though the

zip/unzip tools the user has may be well able to handle it.



=== modified file 'lisp/arc-mode.el'

--- lisp/arc-mode.el    2011-12-15 07:24:10 +0000

+++ lisp/arc-mode.el    2011-12-20 22:32:35 +0000

@@ -748,6 +748,7 @@

   ;; as an archive by other software.

   (let (case-fold-search)

     (cond ((looking-at "\\(PK00\\)?[P]K\003\004") 'zip)

+          ((looking-at "....PK\001\002") 'zip)

          ((looking-at "..-l[hz][0-9ds]-") 'lzh)

          ((looking-at "....................[\334]\247\304\375") 'zoo)

          ((and (looking-at "\C-z")     ; signature too simple, IMHO



I'm including a patch to 24.0.50, which I just checked out. I haven't

been using it on this machine.





--=-=-=

Content-Type: application/octet-stream

Content-Disposition: attachment; filename=zip-magic.bundle

Content-Transfer-Encoding: base64

Content-Description: git commit that fixes this problem



IyB2MiBnaXQgYnVuZGxlCi01NWNlZDNjNzBkNGM2YTVhNTI0OWZkYjk2Y2Y1Nzk3MDI1NTA1ZDk1

IFVwZGF0ZSBwcm9ncmFtcy50ZXhpIGFuZCBjYy1tb2RlLnRleGkgZm9yIG5ldyBDQyBNb2RlIGZl

YXR1cmVzOiBjYy1tb2RlLnRleGk6IFVwZGF0ZSB2ZXJzaW9uIHN0cmluZyB0byA1LjMyLiBwcm9n

cmFtcy50ZXhpOiBEZXNjcmliZSAiZ3Vlc3NpbmciIG9mIGluZGVudGF0aW9uIHN0eWxlLiAgVXBk

YXRlIEMtTS1hL2UuCjU5NTY5MzIzODY4ZGE3ZmE5YmNhNzA0NTdjMDUwZTZhNzZhMjc0ZDggSEVB

RAoKUEFDSwAAAAIAAAAEnzN4nI1SzY6bMBC+8xSjvXS3SigkOISqqqq2W6nnfYLBHsAK2NQesmGf

vgPsqu2tF4TNfL8DByKo83OTV6qpD3hQ6ljkp2OVFWfURyxNnhE1iuqqOCcjBnIMSmkyR11mptAn

VKgORdWYujrpRpVVmQlJpkylEpy48wG+o7PUw9fg/SXCJ1MX59uX9Zk64s+QHw9FccjKLId9ds6y

RPthsMz0v9C8VMdXaPIeehvHDxj0fvCGUkHfy6GzV9o3tqc9zyM9fIRA2rfOvhAgvNgRlm/AHTJ0

GCH6geCuxVBjS3dgHTTBS3TfyAzBgK3VSfLTyclG0BhpvX8FgNyh5gn7fgZ0gmoisbCwX8cWrR3U

E8PF+Wfr2k342bt3Ik/9CI8DauFwcwr3j1cKs+jIWE2ND5vUG2cER2TIgJnCxkTwwwYZvEFkDDyN

MAavKcadxALLEh1N3CSHSXdvodYGXhVk03FhEwfLZwGJde2dWOHlJMm2OAiRfk3yV1jsV+L0Ye1l

FDPajv0/9WrpQkgYpU95lbIsBwzS0eCnrd0/nf+Vda17db/WvWT24kG4BlHgxfothW9dWLaGzsAT

Nhgs0I3JReudRBcyOeMC2MmKryTGJY247iZnghRYz0xx8aDDPLLft5O7gHS0ONj2I5jOT70B2zrx

lia/AeZPGLLxAnpBrnP5aIAs30EMyYc1GQbb9d4YeJzbw7mHc4M5s8itlUIbhTZzboi6y2Qu8rNM

tyfPJmazN3MhIwDcQAxy8QLHto+9iWiS8e5KrDRWSBG/X4NRZXicASEA3v+nV6dXsAMEFMU4z58P

xsIsqwIH/r0RB+FXgat4sxcEkCfxog3e8QKBYpEWb65rYrPXF3hJhDRoSLRz7XicW7OG9fYa1g1f

8icfKJTg0wOCAO8YAwNDIDba/Drf8RkA7XMNSTY08bjoazSA8VXWJlozFu5bfPWk

--=-=-=

If outlook has mangled this badly, I can resend it from my usual computer later.





In GNU Emacs 23.3.1 (i386-mingw-nt5.1.2600)

of 2011-03-10 on 3249CTO

Windowing system distributor `Microsoft Corp.', version 5.1.2600

configured using `configure --with-gcc (4.5) --no-opt --cflags -Ic:/imagesupport/include'



Important settings:

  value of $LC_ALL: nil

  value of $LC_COLLATE: nil

  value of $LC_CTYPE: nil

  value of $LC_MESSAGES: nil

  value of $LC_MONETARY: nil

  value of $LC_NUMERIC: nil

  value of $LC_TIME: nil

  value of $LANG: ENU

  value of $XMODIFIERS: nil

  locale-coding-system: cp1252

  default enable-multibyte-characters: t



Major mode: nXML



Minor modes in effect:

  shell-dirtrack-mode: t

  server-mode: t

  show-paren-mode: t

  recentf-mode: t

  mouse-wheel-mode: t

  file-name-shadow-mode: t

  global-font-lock-mode: t

  font-lock-mode: t

  auto-encryption-mode: t

  auto-compression-mode: t

  line-number-mode: t

  transient-mark-mode: t



Recent input:

<down> <down> <down> <down> <down> <down> <down> <down>

<down> <down> C-h v <C-down-mouse-1> <return> C-h v

<return> M-x c u s t o <tab> - v <tab> <return> a r

c h i v e - z i p - e x t r a c t <return> C-x b o

m n <return> C-x b C-s <return> <up> <up> <up> <up>

<up> <up> <up> <up> <up> <up> <up> C-h f <return> C-x

o <tab> <return> C-x 1 <right> <right> <left> <left>

<down> <down> <left> <left> C-h v C-g C-g <right> <left>

<right> C-h v C-g <C-down-mouse-1> C-h f <return> C-x

o <tab> C-x o <right> <down> <down> <down> C-x o <down>

<down> <down> <down> <right> <right> <right> <right>

<right> <right> <right> <right> <down> <down> <down>

<down> <down> <down> <down> C-x o <down> <left> <left>

<left> <left> C-x o <up> <up> <up> <up> <up> <up> <up>

<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>

<up> <up> C-x o <up> <up> <up> <down> <down> ( t SPC

n i l C-e <backspace> <backspace> C-x C-s C-x b <return>

C-x b <down-mouse-1> <mouse-2> <return> C-g <C-down-mouse-1>

C-g C-x o C-x o C-g C-x b <return> C-x k <return> C-x

b <return> <down> <down> C-e C-x C-e C-x b <return>

<up> <return> C-x <down-mouse-1> <mouse-1> b C-g C-g

C-g C-x b <return> <up> <up> <left> <left> <left> <left>

<left> <left> <left> ' <down> <down> C-e C-x C-e C-x

C-s C-x b <return> <return> <next> <next> <next> <next>

<next> <next> <next> <next> <next> <next> <next> <next>

<next> C-x k <return> C-x 1 C-s p a g e i n f o C-a

<return> <next> <next> <next> <next> <next> <next>

<next> <next> <next> <next> <next> <next> <next> <next>

<next> <next> <next> <next> <C-home> M-x r e p o r

t <tab> <return>



Recent messages:

Quit [5 times]

archive-extract-by-stdout

Symbol's function definition is void: t

Quit [3 times]

archive-extract-by-stdout

Saving file c:/mozilla-build/emacs-23.3/lisp/arc-mode.el...

Wrote c:/mozilla-build/emacs-23.3/lisp/arc-mode.el

call-interactively: End of buffer [2 times]

Mark saved where search started

Mark set



Load-path shadows:

c:/Documents and Settings/brooksd/.emacs.d/maxframe.el/maxframe hides c:/Documents and Settings/brooksd/.emacs.d/maxframe

c:/Documents and Settings/brooksd/.emacs.d/loaddefs hides c:/mozilla-build/emacs-23.3/lisp/loaddefs

c:/Documents and Settings/brooksd/.emacs.d/custom hides c:/mozilla-build/emacs-23.3/lisp/custom

c:/Documents and Settings/brooksd/.emacs.d/elpa/ruby-mode-1.1/ruby-mode hides c:/mozilla-build/emacs-23.3/lisp/progmodes/ruby-mode

c:/Documents and Settings/brooksd/.emacs.d//elpa-to-submit/cperl-mode hides c:/mozilla-build/emacs-23.3/lisp/progmodes/cperl-mode



Features:

(shadow flyspell ispell mail-extr message ecomplete rfc822 mml mml-sec

mm-decode mm-bodies mailabbrev nnheader gmm-utils mailheader canlock

sha1 hex-util hashcash mail-utils emacsbug nxml-uchnm rng-nxml rng-valid

rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn

nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc

xmltok css-mode apropos generic vc-git ebnf2ps ps-print ps-def lpr

epg-config timeclock time-stamp tar-mode sort snmp-mode tempo remember

generic-x forms-mode forms doc-view jka-compr image-mode dired dns-mode

conf-mode newcomment pp cus-edit multi-isearch ruler-mode mule-util hexl

descr-text hi-lock find-func debug help-mode view arc-mode archive-mode

tramp-imap assoc tramp-gw tramp-fish tramp-cache tramp-ftp tramp-cmds

tramp auth-source gnus-util netrc shell comint ring password-cache

format-spec tramp-compat trampver eldoc paredit hl-line

idle-highlight-mode epresent derived org-latex org-export-latex

regexp-opt footnote org-exp org-exp-blocks org-agenda org-clock org

byte-opt warnings bytecomp byte-compile org-footnote org-src org-list

org-faces org-compat org-macs noutline outline easy-mmode server elnode

mm-encode mailcap mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums

mm-util time-date mail-prsvr maxframe linum cus-start cus-load

starter-kit-js starter-kit-ruby starter-kit-perl starter-kit-lisp

starter-kit-eshell starter-kit-registers starter-kit-misc ido paren

starter-kit-bindings windmove edmacro kmacro starter-kit-defuns imenu

thingatpt dominating-file recentf tree-widget wid-edit easymenu

ansi-color uniquify advice help-fns advice-preload ffap saveplace

starter-kit-elpa cl cl-19 find-file-in-project-autoloads gist-autoloads

idle-highlight-autoloads idle-highlight-mode-autoloads

inf-ruby-autoloads magit-autoloads ruby-mode-autoloads

yaml-mode-autoloads package tooltip ediff-hook vc-hooks lisp-float-type

mwheel dos-w32 disp-table ls-lisp w32-win w32-vars tool-bar dnd fontset

image fringe lisp-mode register page menu-bar rfn-eshadow timer select

scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core

frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai

tai-viet lao korean japanese hebrew greek romanian slovak czech european

ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help

simple abbrev loaddefs button minibuffer faces cus-face files

text-properties overlay md5 base64 format env code-pages mule custom

widget hashtable-print-readable backquote make-network-process multi-tty

emacs)



--=-=-=--*** E-Mail body has been placed on clipboard, please paste it here! ***

[Message part 2 (text/html, inline)]

Message #8 received at 10338 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> jurta.org>
To: "Brooks\, Daniel" <Daniel.Brooks <at> ask.com>
Cc: 10338 <at> debbugs.gnu.org
Subject: Re: bug#10338: 23.3;
	Firefox includes a jar file which is not recognied as a zip file
	(omni.jar)
Date: Thu, 22 Dec 2011 22:02:20 +0200

> They've optimized for read performance by rearranging the sections of
> the file (and the files inside the archive). As a result the magic is
> slightly different and Emacs doesn't recognize the file, even though the
> zip/unzip tools the user has may be well able to handle it.

Here is what the author of InfoZip says about this non-standard jar format
in https://bugzilla.mozilla.org/show_bug.cgi?id=605524#c11 :

  - Fifth, the first 10 bytes of this archive are (in hex):
           b8    b6    14    00    50    4b    01    02    17    0b
                                   P     K     1     2

  The P K 1 2 is the first Central Directory signature.  The 4 bytes
  before that appear to be some index.  Using little endian math, this
  ends up being 1357496, which happens to be the offset to local record
  #210, a .png file.  Not sure if this is what it is or why this is
  useful.  However, if everyone starts tucking their own bytes into the
  .zip structure without using the established procedures for doing that,
  I think it's asking for chaos.  The standard allows leading bytes, but
  usually the EOCDR can point a utility to the beginning of the real
  archive data, but in this case the EOCDR is buried where no one can find
  it without scanning the archive.

For testing purposes such jar files can be produced with the following utility:

  http://hg.mozilla.org/mozilla-central/raw-file/default/config/optimizejars.py

> === modified file 'lisp/arc-mode.el'
> --- lisp/arc-mode.el    2011-12-15 07:24:10 +0000
> +++ lisp/arc-mode.el    2011-12-20 22:32:35 +0000
> @@ -748,6 +748,7 @@
>    ;; as an archive by other software.
>    (let (case-fold-search)
>      (cond ((looking-at "\\(PK00\\)?[P]K\003\004") 'zip)
> +          ((looking-at "....PK\001\002") 'zip)
>           ((looking-at "..-l[hz][0-9ds]-") 'lzh)
>           ((looking-at "....................[\334]\247\304\375") 'zoo)
>           ((and (looking-at "\C-z")     ; signature too simple, IMHO
>
> I'm including a patch to 24.0.50, which I just checked out. I haven't
> been using it on this machine.

Thanks, your patch makes reading of non-standard jar files more permissive.

There are two other places with the regexps for ZIP headers in
`magic-fallback-mode-alist' and `auto-coding-regexp-alist'
intended to read ZIP archives with non-standard filename extensions.
Without adding "....PK\001\002" to them too, you won't able to open the
"omni.ja" file (another non-standard filename extension from Mozilla).
But perhaps this is not necessary - it would be too much to allow
reading of files with both non-standard filename extensions and
non-standard headers at the same time.

More important problem is that most archivers don't understand this
format, so after opening such files, extra lines are inserted into the
file buffer:

  warning [/tmp/omni.jar]:  6552040 extra bytes at beginning or within zipfile
    (attempting to process anyway)
  error [/tmp/omni.jar]:  reported length of central directory is
    -6552040 bytes too long (Atari STZip zipfile?  J.H.Holm ZIPSPLIT 1.1
    zipfile?).  Compensating...

It seems this problem prompted the bug#10347.

Message #11 received at 10338 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> jurta.org>
To: 10338 <at> debbugs.gnu.org
Subject: Re: bug#10338: 23.3;
	Firefox includes a jar file which is not recognied as a zip file
	(omni.jar)
Date: Thu, 22 Dec 2011 22:28:48 +0200

I found another problem with Firefox jar files: trying to visit the file
`browser.jar' doesn't use `archive-mode'.  It's visited in `html-mode'.
That's because the first line of the ZIP-based JAR archive contains
the file-local variable -*- line from the first file in this archive:

  PK^C^D content/browser/aboutDialog.xul<?xml version="1.0"?> <!-- -*- Mode: HTML -*- -->

This is purely Emacs problem, not caused by something non-standard from Mozilla.
We should somehow change the priorities of detecting the correct mode.

After filling the -*- line with spaces in hexl-mode, the jar file opens
normally, but visiting a file from the archive inserts the following line into
the file buffer:

  content/browser/aboutDialog.xul  bad CRC 1ded366d  (should be b80ec225)

This will be fixed in bug#10347 by displaying the warning and error messages
in the echo area instead of inserting into the buffer.

Message #14 received at 10338 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> jurta.org>
To: 10338 <at> debbugs.gnu.org
Subject: Re: bug#10338: 23.3;
	Firefox includes a jar file which is not recognied as a zip file
	(omni.jar)
Date: Fri, 28 Sep 2012 19:40:43 +0300

tags 10338 wontfix
close 10338
stop

After reading the ongoing discussion at

https://bugzilla.mozilla.org/show_bug.cgi?id=605524

it doesn't seem to make sense to add special processing
for just one file omni.ja with a broken format that is likely
to be obsolete soon.  This report could be reopened in the case
when this unusual format becomes more wide-spread.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.