GNU bug report logs -
#40236
[PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.
Previous Next
Reported by: Pierre Neidhardt <mail <at> ambrevar.xyz>
Date: Thu, 26 Mar 2020 08:36:01 UTC
Severity: normal
Tags: patch
Done: Ludovic Courtès <ludo <at> gnu.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40236 in the body.
You can then email your comments to 40236 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 26 Mar 2020 08:36:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Pierre Neidhardt <mail <at> ambrevar.xyz>
:
New bug report received and forwarded. Copy sent to
guix-patches <at> gnu.org
.
(Thu, 26 Mar 2020 08:36:01 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
* gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
---
gnu/system/examples/desktop.tmpl | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
index 3931bad60d..e61e8064cd 100644
--- a/gnu/system/examples/desktop.tmpl
+++ b/gnu/system/examples/desktop.tmpl
@@ -34,7 +34,8 @@
(list (file-system
(device (file-system-label "my-root"))
(mount-point "/")
- (type "ext4")
+ (type "btrfs")
+ (options "subvol=rootfs,compress=zstd")
(dependencies mapped-devices))
(file-system
(device (uuid "1234-ABCD" 'fat))
--
2.25.1
Added blocking bug(s) 37305
Request was from
Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Tue, 31 Mar 2020 01:28:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 31 Mar 2020 01:53:01 GMT)
Full text and
rfc822 format available.
Message #10 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi!
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
> ---
> gnu/system/examples/desktop.tmpl | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
> index 3931bad60d..e61e8064cd 100644
> --- a/gnu/system/examples/desktop.tmpl
> +++ b/gnu/system/examples/desktop.tmpl
> @@ -34,7 +34,8 @@
> (list (file-system
> (device (file-system-label "my-root"))
> (mount-point "/")
> - (type "ext4")
> + (type "btrfs")
> + (options "subvol=rootfs,compress=zstd")
> (dependencies mapped-devices))
> (file-system
> (device (uuid "1234-ABCD" 'fat))
I think Btrfs with compression is a fine, modern default of a file
system, but we shall get a good samples of opinions (I expect a variety
of them :-)) and common agreement before pushing this change.
For those wondering about the benefits of having the root file system on
a subvolume (named 'rootfs' in the proposed configuration), the
following page [0] explains it as:
[...] The above layout, which obviously serves as the system's
"main" filesystem, places data directly within the top-level
subvolume (namely everything for example /usr, that's not in a
child subvolume) This makes changing the structure (for example to
something more flat) more difficult, which is why it's generally
suggested to place the actual data in a subvolume (that is not the
top-level subvolume), in the above example, a better layout would
be the following:
toplevel (volume root directory, not mounted)
\-- root (subvolume root directory, to be mounted at /)
+-- home (subvolume root directory)
+-- var (subvolume root directory)
+-- www (subvolume root directory)
+-- lib (directory)
\-- postgresql (subvolume root directory)
In short, it allows exposing just the subvolumes which should be
visible, instead of everything.
I've marked this blocked by 37305, which includes GRUB support for
booting from a subvolume as well as documentation about Btrfs usage in
Guix.
[0] https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 31 Mar 2020 07:53:02 GMT)
Full text and
rfc822 format available.
Message #13 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thanks!
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 31 Mar 2020 12:10:01 GMT)
Full text and
rfc822 format available.
Message #16 received at submit <at> debbugs.gnu.org (full text, mbox):
On 26.03.20 09:35, Pierre Neidhardt wrote:
> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
I would oppose this change as I had too much troubles with Btrfs on my
openSUSE machines. I used a simple partition layout spanning the whole
disk. AFAIK this is not really recommended with btrfs. My laptop was
from time to time unusable until I reinstalled it with ext4...
Further do we need all this rollback stuff from btrfs if we have it
already in Guix?
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 31 Mar 2020 14:55:01 GMT)
Full text and
rfc822 format available.
Message #19 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:
>> (list (file-system
>> (device (file-system-label "my-root"))
>> (mount-point "/")
>> - (type "ext4")
>> + (type "btrfs")
>> + (options "subvol=rootfs,compress=zstd")
By the way in your patch you did not document the syntax (options
"subvol=...").
Is it supported nonetheless in your patch? Is it supported in current Guix?
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 31 Mar 2020 23:21:02 GMT)
Full text and
rfc822 format available.
Message #22 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello Pierre!
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:
>
>>> (list (file-system
>>> (device (file-system-label "my-root"))
>>> (mount-point "/")
>>> - (type "ext4")
>>> + (type "btrfs")
>>> + (options "subvol=rootfs,compress=zstd")
>
> By the way in your patch you did not document the syntax (options
> "subvol=...").
Are you sure you are looking at the right patch series
(https://issues.guix.info/issue/37305#16)? I documented usage examples
for a root file system using Btrfs subvolumes.
> Is it supported nonetheless in your patch?
Yes.
> Is it supported in current Guix?
Not currently; the patch set is in review :-). I have it running on 3
machines and there's a system test which shows it working as well, so
don't be afraid to try it!
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Wed, 01 Apr 2020 07:01:01 GMT)
Full text and
rfc822 format available.
Message #25 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:
>> By the way in your patch you did not document the syntax (options
>> "subvol=...").
>
> Are you sure you are looking at the right patch series
> (https://issues.guix.info/issue/37305#16)? I documented usage examples
> for a root file system using Btrfs subvolumes.
I was looking at the wrong patch indeed, sorry about that! :)
>> Is it supported nonetheless in your patch?
>
> Yes.
>
>> Is it supported in current Guix?
>
> Not currently; the patch set is in review :-). I have it running on 3
> machines and there's a system test which shows it working as well, so
> don't be afraid to try it!
Does this go on master or core-updates?
Does it rebuild the world?
I don't have much hardware to build anything at the moment, so I can only test
this if it's a small build ;)
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Wed, 01 Apr 2020 21:29:01 GMT)
Full text and
rfc822 format available.
Message #28 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi!
Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
> I've marked this blocked by 37305, which includes GRUB support for
> booting from a subvolume as well as documentation about Btrfs usage in
> Guix.
Also, the installer defaults to ext4 and that’s probably a more tested
configuration currently.
Ludo’.
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 02 Apr 2020 07:16:01 GMT)
Full text and
rfc822 format available.
Message #31 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes:
> Also, the installer defaults to ext4 and that’s probably a more tested
> configuration currently.
Could a subvolume-less Btrfs be more problematic than ext4?
If not, it might be a good idea to also switch to Btrfs in the installer.
I've tested Btrfs with Zstd:3 compression on a new GNOME + EXWM install:
I've achieved a stunning 30% compression like Maxim.
I must say that it's extremely useful, if not necessary, on a 60GiB SSD... :D
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 02 Apr 2020 08:05:01 GMT)
Full text and
rfc822 format available.
Message #34 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi,
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:
> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Also, the installer defaults to ext4 and that’s probably a more tested
>> configuration currently.
>
> Could a subvolume-less Btrfs be more problematic than ext4?
> If not, it might be a good idea to also switch to Btrfs in the installer.
I think it’s more work than just switching: we have to do enough
testing, add new automated tests, etc.
Also, any change in this area should happen post-release. :-)
Thanks,
Ludo’.
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 02 Apr 2020 10:37:01 GMT)
Full text and
rfc822 format available.
Message #37 received at submit <at> debbugs.gnu.org (full text, mbox):
On 02.04.20 10:04, Ludovic Courtès wrote:
>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.
>
> Also, any change in this area should happen post-release. :-)
Why not adding a gnu/system/examples/btrfs.tmpl in the mean time. There
you could showcase all the stuff btrfs can do for us :)
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sat, 04 Apr 2020 01:29:02 GMT)
Full text and
rfc822 format available.
Message #40 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello,
Ludovic Courtès <ludo <at> gnu.org> writes:
> Hi,
>
> Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:
>
>> Ludovic Courtès <ludo <at> gnu.org> writes:
>>
>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.
Fine points.
> Also, any change in this area should happen post-release. :-)
Agreed!
Let's keep this issue open and revisit what extra tests and validation
should be done after the release to consider a switch viable.
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 06 Apr 2020 20:21:02 GMT)
Full text and
rfc822 format available.
Message #43 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Jonathan,
For some reason I did not receive your email:
Jonathan Brielmaier wrote on Tue Mar 31 14:09:04+0200 2020
> > * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
>
> I would oppose this change as I had too much troubles with Btrfs on my
> openSUSE machines. I used a simple partition layout spanning the whole
> disk. AFAIK this is not really recommended with btrfs. My laptop was
> from time to time unusable until I reinstalled it with ext4...
This is for Guix System only. Did you mean that it would be problematic
if you were running openSUSE in dual boot?
> Further do we need all this rollback stuff from btrfs if we have it
> already in Guix?
Btrfs has many benefits over Ext3:
it offers compression to about 30% on average, it supports subvolumes,
snapshots and snapshot syncing, and much more.
For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 06 Apr 2020 20:43:01 GMT)
Full text and
rfc822 format available.
Message #46 received at 40236 <at> debbugs.gnu.org (full text, mbox):
On 06.04.20 22:20, Pierre Neidhardt wrote:> Jonathan Brielmaier wrote on
Tue Mar 31 14:09:04+0200 2020
>>> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
>>
>> I would oppose this change as I had too much troubles with Btrfs on my
>> openSUSE machines. I used a simple partition layout spanning the whole
>> disk. AFAIK this is not really recommended with btrfs. My laptop was
>> from time to time unusable until I reinstalled it with ext4...
>
> This is for Guix System only. Did you mean that it would be problematic
> if you were running openSUSE in dual boot?
No, I did run openSUSE on btrfs, there was no Guix involved at all. But
btrfs seemed to be the root case of all my troubles (performance,
hang-ups etc).
>> Further do we need all this rollback stuff from btrfs if we have it
>> already in Guix?
>
> Btrfs has many benefits over Ext3:
>
> it offers compression to about 30% on average, it supports subvolumes,
> snapshots and snapshot syncing, and much more.
Snapshots did fill up my disk. I had no use for them on my laptop. On a
Guix System even less, because you have rollbacks from the package
manager :)
> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)
So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
rust etc :P
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 07 Apr 2020 07:09:02 GMT)
Full text and
rfc822 format available.
Message #49 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Jonathan Brielmaier <jonathan.brielmaier <at> web.de> writes:
> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
> btrfs seemed to be the root case of all my troubles (performance,
> hang-ups etc).
When did you try this? Maybe some issues have been fixed in the
meantime.
I've used Btrfs on a few computers for many months now and it's been a
bliss. Of course experiences vary ;)
>>> Further do we need all this rollback stuff from btrfs if we have it
>>> already in Guix?
>>
>> Btrfs has many benefits over Ext3:
>>
>> it offers compression to about 30% on average, it supports subvolumes,
>> snapshots and snapshot syncing, and much more.
>
> Snapshots did fill up my disk.
What do you mean?
> I had no use for them on my laptop. On a
> Guix System even less, because you have rollbacks from the package
> manager :)
Snapshots are not just for rollbacks: you can sync them across the
network to a remote machine. This allows you to deploy disk images very
fast.
>> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)
>
> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
> rust etc :P
In the end, what I'm suggesting is this issue is merely a
recommendation.
Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
64 GiB SSD. With compression on, you suddenly get 3x more space for
your /gnu/store :)
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Wed, 08 Apr 2020 03:19:01 GMT)
Full text and
rfc822 format available.
Message #52 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello,
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Jonathan Brielmaier <jonathan.brielmaier <at> web.de> writes:
>
>> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
>> btrfs seemed to be the root case of all my troubles (performance,
>> hang-ups etc).
I have yet to encounter this kind of problem on the 3 Guix Systems I've
installed with root Btrfs file systems. It's been rock stable, even
under heavy use (I have a Guix machine configured as a Jenkins slave at
work that builds Yocto projects -- it churns through GiB of files
daily).
[...]
>> Snapshots did fill up my disk.
Snapshots only fill up the disk when we use them (and leave them behind
for enough time that the content they refer to has been mostly
rewritten.
>> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
>> rust etc :P
>
> In the end, what I'm suggesting is this issue is merely a
> recommendation.
>
> Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> 64 GiB SSD. With compression on, you suddenly get 3x more space for
> your /gnu/store :)
I agree that compression is a nice feature.. It also speeds sequential
disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
ext4, I have to 'guix gc' too often, and worry a lot about spaces
(there's literally not much else than Guix on the drive, but it manages
to fill it up quite easily :-).
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 09 Apr 2020 20:14:02 GMT)
Full text and
rfc822 format available.
Message #55 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Tue, Apr 07, 2020 at 11:18:40PM -0400, Maxim Cournoyer wrote:
> Hello,
>
> Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
>
> > Jonathan Brielmaier <jonathan.brielmaier <at> web.de> writes:
> >
> >> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
> >> btrfs seemed to be the root case of all my troubles (performance,
> >> hang-ups etc).
>
> I have yet to encounter this kind of problem on the 3 Guix Systems I've
> installed with root Btrfs file systems. It's been rock stable, even
> under heavy use (I have a Guix machine configured as a Jenkins slave at
> work that builds Yocto projects -- it churns through GiB of files
> daily).
>
> [...]
>
> >> Snapshots did fill up my disk.
>
> Snapshots only fill up the disk when we use them (and leave them behind
> for enough time that the content they refer to has been mostly
> rewritten.
>
> >> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
> >> rust etc :P
> >
> > In the end, what I'm suggesting is this issue is merely a
> > recommendation.
> >
> > Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> > 64 GiB SSD. With compression on, you suddenly get 3x more space for
> > your /gnu/store :)
>
> I agree that compression is a nice feature.. It also speeds sequential
> disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
> ext4, I have to 'guix gc' too often, and worry a lot about spaces
> (there's literally not much else than Guix on the drive, but it manages
> to fill it up quite easily :-).
>
(ins)efraim <at> E5400 ~$ sudo compsize -x /gnu/store/
Password:
Processed 3158140 files, 737675 regular extents (2467369 refs), 1335101 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 74% 51G 69G 180G
none 100% 32G 32G 86G
lzo 51% 19G 36G 93G
My understanding of this is that I have 36GB of files that are
compressed at 51% to 19GB, and overall due to the deduplication in the
store I have references to what would otherwise be 180GB total taking up
only 51G.
So compression saves me 26% ([69-51]/69), and deduplication saves me
62% ([180-69]/180).
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Fri, 10 Apr 2020 07:40:02 GMT)
Full text and
rfc822 format available.
Message #58 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> So compression saves me 26% ([69-51]/69), and deduplication saves me
> 62% ([180-69]/180).
Thanks for sharing!
zstd might give better results. Any reason you chose lzo over zstd?
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Fri, 10 Apr 2020 08:26:01 GMT)
Full text and
rfc822 format available.
Message #61 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
> > So compression saves me 26% ([69-51]/69), and deduplication saves me
> > 62% ([180-69]/180).
>
> Thanks for sharing!
> zstd might give better results. Any reason you chose lzo over zstd?
>
My machine is about 10 years old so I was more concerned than normal
about the CPU usage. If lz4 was an option I would've gone with that, but
according to the Arch wiki or some other locations lzo was basically the
fastest option.
Since mail is mostly text I occasionally recompress that. I haven't
figured out how to use 'btrfs filesystem defragment' to choose the
compression level for zstd.
(ins)efraim <at> E5400 ~$ sudo compsize Maildir/
Password:
Processed 121378 files, 129245 regular extents (129245 refs), 19963 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 66% 3.3G 4.9G 4.9G
none 100% 282M 282M 282M
zlib 55% 137K 248K 248K
lzo 53% 125M 233M 233M
zstd 64% 2.9G 4.4G 4.4G
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Fri, 10 Apr 2020 09:05:01 GMT)
Full text and
rfc822 format available.
Message #64 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Efraim Flashner <efraim <at> flashner.co.il> writes:
> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>> > 62% ([180-69]/180).
>>
>> Thanks for sharing!
>> zstd might give better results. Any reason you chose lzo over zstd?
>>
>
> My machine is about 10 years old so I was more concerned than normal
> about the CPU usage. If lz4 was an option I would've gone with that, but
> according to the Arch wiki or some other locations lzo was basically the
> fastest option.
I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly
smooth, can't notice any performance drop. In fact, I wonder if it's
not even faster than before, but it's hard to measure.
Note that Arch Wiki tends to be on the conservative side when it comes
to performance. I would not use it as a reference for the general case:
it may guide users to sacrifice convenience and features over
unnoticeable performance gains.
Cheers!
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 14 Apr 2020 02:22:02 GMT)
Full text and
rfc822 format available.
Message #67 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi!
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Efraim Flashner <efraim <at> flashner.co.il> writes:
>
>> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>>> > 62% ([180-69]/180).
>>>
>>> Thanks for sharing!
>>> zstd might give better results. Any reason you chose lzo over zstd?
>>>
>>
>> My machine is about 10 years old so I was more concerned than normal
>> about the CPU usage. If lz4 was an option I would've gone with that, but
>> according to the Arch wiki or some other locations lzo was basically the
>> fastest option.
>
> I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly
> smooth, can't notice any performance drop. In fact, I wonder if it's
> not even faster than before, but it's hard to measure.
I've also tried zstd (default level, 3) on a Intel Q6700 desktop (2007).
I don't see any CPU spike caused by the compression. It's operating
quite smoothly actually, and gives me the following space savings:
--8<---------------cut here---------------start------------->8---
sudo compsize -x /gnu/store
Processed 1613194 files, 402674 regular extents (1163093 refs), 665696 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 60% 15G 25G 63G
none 100% 10G 10G 28G
zstd 33% 5.1G 15G 34G
--8<---------------cut here---------------end--------------->8---
Recently on the #btrfs channel someone suggested to use the Btrfs option
compress-force=zstd rather than compress=zstd, the reason being that
zstd has its own algorithm to determine if it should compress a file or
not, and that this is faster than what Btrfs does on its own when trying
to test for compressibility.
Another suggestion was to use space_cache=v2 (it defaults to v1). This
is supposedly more efficient at managing the free space pool on large
drives (TB and up).
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 14 Apr 2020 06:54:01 GMT)
Full text and
rfc822 format available.
Message #70 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thanks for sharing those tips!
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sat, 02 May 2020 13:31:03 GMT)
Full text and
rfc822 format available.
Message #73 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Maxim,
Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)
going to make it on the next core-updates merge or is it too late?
Cheers!
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sat, 02 May 2020 13:51:01 GMT)
Full text and
rfc822 format available.
Message #76 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Hi Maxim,
>
> Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)
> going to make it on the next core-updates merge or is it too late?
Why would it have to go through 'core-updates'?
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sat, 02 May 2020 13:59:01 GMT)
Full text and
rfc822 format available.
Message #79 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I assumed it rebuilt the world, maybe I remembered wrong.
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sat, 02 May 2020 19:04:02 GMT)
Full text and
rfc822 format available.
Message #82 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> I assumed it rebuilt the world, maybe I remembered wrong.
No, it doesn't :-). After it gets duly reviewed, it can go to the
master branch.
I've recently had the "chance" to experience a degraded RAID1 array,
which made me reconsider some of the simplifications that were made to
the original patch series (such as removing the option for users to pass
root options to the initrd -- this would be useful to input the
"degraded" mount option required to boot a degraded array, if your
system is no longer in a capacity to be reconfigured). So I may want to
reintroduce those changes, although this could also be done at a later
point as well.
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sun, 03 May 2020 07:02:02 GMT)
Full text and
rfc822 format available.
Message #85 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
OK, thanks for the details.
What is "degraded RAID1"?
I'll see if I can test. the patch on my Btrfs system.
Anything else I can do to help?
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 04 May 2020 15:24:02 GMT)
Full text and
rfc822 format available.
Message #88 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> OK, thanks for the details.
>
> What is "degraded RAID1"?
Degraded RAID1 would be a RAID1 array (mirror disks) in which at least
one drive has failed. As long as at least a single healthy disk
remains, the RAID1 array should remain functional but requires to be
mounted with the 'degraded' mount option (this is done to flag that
there is a serious problem that needs to be addressed, e.g., replace the
faulty drive).
> I'll see if I can test. the patch on my Btrfs system.
> Anything else I can do to help?
Testing on a live system should provide extra confidence that the patch
work as advertised. I have done so on my machines for a while (there's
also a system test), but I'd like to know if it works for someone else
:-).
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sun, 31 May 2020 07:40:01 GMT)
Full text and
rfc822 format available.
Message #91 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Now that 37305 is merged, we can revisit this issue.
1. This patch only includes a documentation update.
2. We could make Btrfs the default in the graphical installer.
3. We would probably need to update the tests, at least for the latter.
As mentioned above, I think it's safe to enable Btrfs without subvolume,
with Zstd compression.
For subvolumes, we would need to implement the corresponding tests.
Thoughts?
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sun, 31 May 2020 07:57:01 GMT)
Full text and
rfc822 format available.
Message #94 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:
> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
>
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
>
> For subvolumes, we would need to implement the corresponding tests.
>
> Thoughts?
My main concern is the possibility of data loss. I don't know how much
is FUD and how much is legit so to me it seems safest to see what
OpenSuSE uses and to mimic that a bit (in terms of not using most of the
features available).
That said, I use btrfs with no subvolumes with compression turned on and
I'm pretty happy with that.
Two more things:
/var/guix/db should probably have CoW disabled, as should /tmp
would the deduplication of btrfs be "better" than the deduplication from
the daemon?
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Sun, 31 May 2020 21:08:01 GMT)
Full text and
rfc822 format available.
Message #97 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi Pierre,
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:
> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
>
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
>
> For subvolumes, we would need to implement the corresponding tests.
>
> Thoughts?
Like I wrote earlier, ext4 is better tested configuration. For the
people who take care of the release, believe me, it’s reassuring to have
one less moving piece.
So the question is not whether Btrfs is “better”, it’s more about the
confidence and peace of mind of those responsible for the release and
subsequent maintenance.
Now, contributions that make Btrfs well-supported in the installer (is
anything missing there?) and in system tests would be welcome I guess.
In the manual, it’s also fine IMO to mention something like:
Some file systems, such as Btrfs, support compression, which is
reported to nicely complement file deduplication that the daemon
performs independently of the file system (@pxref{Invoking
guix-daemon}).
So a hint for savvy people rather than a recommendation.
Ludo’.
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 04:22:01 GMT)
Full text and
rfc822 format available.
Message #100 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello!
Efraim Flashner <efraim <at> flashner.co.il> writes:
> On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> My main concern is the possibility of data loss. I don't know how much
> is FUD and how much is legit so to me it seems safest to see what
> OpenSuSE uses and to mimic that a bit (in terms of not using most of the
> features available).
Except for Btrfs RAID 5 and 6, which are still known to have issues and
are considered experimental, I'd say mostly FUD, although there were
some bugs in Linux 5.1 and 5.2. As you noted, (Open)SUSE defaults to
Btrfs, and companies such as Synology ship network storage products with
Btrfs on. If it was that bad, nobody would buy those, and the companies
would stop proudly advertising their use of Btrfs.
> That said, I use btrfs with no subvolumes with compression turned on and
> I'm pretty happy with that.
>
> Two more things:
> /var/guix/db should probably have CoW disabled, as should /tmp
I haven't bothered and my system seems to be doing OK. When I asked in
#btrfs, people told me to keep CoW unless I was really sure it was a
problem (i.e., run benchmarks), as it implies loosing the checksum
validation and compression. The command 'man 5 btrfs' also states that
"Updates in-place improve performance for workloads that do frequent
overwrites, at the cost of potential partial writes, in case the write
is interrupted (system crash, device failure).", which doesn't sound
safe to do for something as important as /var/guix/db.
> would the deduplication of btrfs be "better" than the deduplication from
> the daemon?
On my system (with zstd compression), compsize -x /gnu/store suggests
a resounding yes:
--8<---------------cut here---------------start------------->8---
sudo compsize -x /gnu/store
Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 57% 51G 88G 217G
none 100% 32G 32G 81G
zstd 33% 18G 56G 135G
--8<---------------cut here---------------end--------------->8---
The delta between the Uncompressed and Referenced column is attributed
to the deduplication done by Btrfs, and provides massive space savings
in my case (this is just for /gnu/store).
I'd need 217 GiB over a traditional fs such as EXT4 to hold my current
store, while an uncompressed Btrfs partition would use only 88 GiB.
With zstd compression, it's down to 51 GiB, or less that a quarter of
what would have been required using EXT4.
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 04:50:01 GMT)
Full text and
rfc822 format available.
Message #103 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello Pierre,
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
Is it already possible to choose Btrfs in the graphical installer? I
guess so, but I haven't tried.
> 3. We would probably need to update the tests, at least for the
> latter.
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
My personal experience also tells me so, but it is true that it is still
newer and more exotic than EXT4.
> For subvolumes, we would need to implement the corresponding tests.
What kind of tests are you referring to? There is a test in (gnu tests
install) that install root on a Btrfs subvolume and boot from it, but
I'm pretty sure the graphical installer doesn't have support for
configuring a Btrfs subvolumes layout.
> Thoughts?
I'm in favor of encouraging the use of Btrfs with compression given that
Guix puts more stress on storage space compared to traditional distros,
and that I've had only positive experiences with Btrfs in the last year
and half or so that I've used it on my machines.
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 05:04:01 GMT)
Full text and
rfc822 format available.
Message #106 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hello Ludovic,
Ludovic Courtès <ludo <at> gnu.org> writes:
> Hi Pierre,
>
> Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:
>
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> Like I wrote earlier, ext4 is better tested configuration. For the
> people who take care of the release, believe me, it’s reassuring to have
> one less moving piece.
I believe you! :-)
> So the question is not whether Btrfs is “better”, it’s more about the
> confidence and peace of mind of those responsible for the release and
> subsequent maintenance.
>
> Now, contributions that make Btrfs well-supported in the installer (is
> anything missing there?) and in system tests would be welcome I guess.
I don't know about the installer (I guess there's no support for
configuring Btrfs subvolumes graphically), but for system tests we're
already covered (we have a general Btrfs installation test, as well as
one on a subvolume, even a Btrfs RAID one in patches).
> In the manual, it’s also fine IMO to mention something like:
>
> Some file systems, such as Btrfs, support compression, which is
> reported to nicely complement file deduplication that the daemon
> performs independently of the file system (@pxref{Invoking
> guix-daemon}).
>
> So a hint for savvy people rather than a recommendation.
That's reasonable as a first step in democratizing Btrfs usage a bit,
without taking any risk, if that's better for your peace of mind :-).
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 06:17:01 GMT)
Full text and
rfc822 format available.
Message #109 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> >
> > Two more things:
> > /var/guix/db should probably have CoW disabled, as should /tmp
>
> I haven't bothered and my system seems to be doing OK. When I asked in
> #btrfs, people told me to keep CoW unless I was really sure it was a
> problem (i.e., run benchmarks), as it implies loosing the checksum
> validation and compression. The command 'man 5 btrfs' also states that
> "Updates in-place improve performance for workloads that do frequent
> overwrites, at the cost of potential partial writes, in case the write
> is interrupted (system crash, device failure).", which doesn't sound
> safe to do for something as important as /var/guix/db.
Fair enough. I had heard that the CoW stuff wasn't great for databases.
I thought Leo ran into some issues with CoW on /tmp with the syncthing
tests.
> > would the deduplication of btrfs be "better" than the deduplication from
> > the daemon?
>
> On my system (with zstd compression), compsize -x /gnu/store suggests
> a resounding yes:
>
> --8<---------------cut here---------------start------------->8---
> sudo compsize -x /gnu/store
> Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.
> Type Perc Disk Usage Uncompressed Referenced
> TOTAL 57% 51G 88G 217G
> none 100% 32G 32G 81G
> zstd 33% 18G 56G 135G
> --8<---------------cut here---------------end--------------->8---
>
> The delta between the Uncompressed and Referenced column is attributed
> to the deduplication done by Btrfs, and provides massive space savings
> in my case (this is just for /gnu/store).
>
> I'd need 217 GiB over a traditional fs such as EXT4 to hold my current
> store, while an uncompressed Btrfs partition would use only 88 GiB.
> With zstd compression, it's down to 51 GiB, or less that a quarter of
> what would have been required using EXT4.
I always understood that as with compression you're using 51G instead of
88G, and because of deduplication from the daemon it would only be 88G
instead of 217G. I took the numbers from 'none' to mean that the daemon
itself already did a lot of deduplication.
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 07:50:01 GMT)
Full text and
rfc822 format available.
Message #112 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Efraim Flashner <efraim <at> flashner.co.il> writes:
> I always understood that as with compression you're using 51G instead of
> 88G, and because of deduplication from the daemon it would only be 88G
> instead of 217G. I took the numbers from 'none' to mean that the daemon
> itself already did a lot of deduplication.
That's what I understood as well.
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Mon, 01 Jun 2020 18:30:01 GMT)
Full text and
rfc822 format available.
Message #115 received at 40236 <at> debbugs.gnu.org (full text, mbox):
Hi!
Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
> Efraim Flashner <efraim <at> flashner.co.il> writes:
>
>> I always understood that as with compression you're using 51G instead of
>> 88G, and because of deduplication from the daemon it would only be 88G
>> instead of 217G. I took the numbers from 'none' to mean that the daemon
>> itself already did a lot of deduplication.
>
> That's what I understood as well.
tl;dr Your understanding is correct.
I was under the impression that hard links shouldn't inflate the numbers
reported under the "Referenced" column.
I just asked over #btrfs and while hard links shouldn't inflate those
numbers (as hard links aren't reflinks), they currently do. It's a bug.
Maxim
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Tue, 02 Jun 2020 13:38:02 GMT)
Full text and
rfc822 format available.
Message #118 received at 40236 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> Some file systems, such as Btrfs, support compression, which is
> reported to nicely complement file deduplication that the daemon
> performs independently of the file system (@pxref{Invoking
> guix-daemon}).
Looks good to me! Do you want to proceed and add it then?
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
Reply sent
to
Ludovic Courtès <ludo <at> gnu.org>
:
You have taken responsibility.
(Wed, 03 Jun 2020 20:02:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Pierre Neidhardt <mail <at> ambrevar.xyz>
:
bug acknowledged by developer.
(Wed, 03 Jun 2020 20:02:02 GMT)
Full text and
rfc822 format available.
Message #123 received at 40236-done <at> debbugs.gnu.org (full text, mbox):
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:
>> Some file systems, such as Btrfs, support compression, which is
>> reported to nicely complement file deduplication that the daemon
>> performs independently of the file system (@pxref{Invoking
>> guix-daemon}).
>
> Looks good to me! Do you want to proceed and add it then?
I pushed something in b6c445931e85c07286a79bc9c80fdc956d58b1e2.
I suggest we revisit the question of the default/recommended file system
in several months, a year or so.
Thanks,
Ludo’.
Information forwarded
to
guix-patches <at> gnu.org
:
bug#40236
; Package
guix-patches
.
(Thu, 04 Jun 2020 09:18:02 GMT)
Full text and
rfc822 format available.
Message #126 received at 40236-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thanks!
--
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Thu, 02 Jul 2020 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 3 years and 289 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.