workstation/tickets
13
0
Issues 25 Activity

enable transparent compression by default #177

New issue
Closed
opened 2020-08-20 01:05:56 +00:00 by chrismurphy · 12 comments
chrismurphy commented 2020-08-20 01:05:56 +00:00
Owner
Copy link

An optional feature for Fedora 33 Btrfs by default proposal is transparent compression.

Summary of benefits:

  • saves space
  • reduces write amplification -> increases flash lifespan
  • can increase read and write performance

Summary of ways to enable compression:

  • mount option -o compress=zstd:1 enables it on the entire file system
  • per directory (xattr to enable, nocompress flag to disable)
  • can be used separately or in combination

Example 1:

  • Facebook is using mount option -o compress=zstd:1across the board wherever they use Btrfs, no exceptions.

Example 2:

  • Installer sets mount option -o compress=zstd:1
  • Installer sets NOCOMPRESS on /home
  • user enables it per directory or file using an xattr, via btrfs property set or exposed by desktop integration in the GUI

A call for testing could be useful in helping to make decisions, and provide some "how to" documentation to encourage it.

The generic issue and technical concerns of actually implementing this across Fedora, are in fedora-btrfs / project: enable compression on btrfs installs

This issue for Workstation Edition, to discuss UI/UX and integration related issues.

An optional feature for Fedora 33 *Btrfs by default proposal* is [transparent compression.](https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Compression) Summary of benefits: - saves space - reduces write amplification -> increases flash lifespan - can increase read and write performance Summary of ways to enable compression: - mount option `-o compress=zstd:1` enables it on the entire file system - per directory (xattr to enable, nocompress flag to disable) - can be used separately or in combination Example 1: - Facebook is using mount option `-o compress=zstd:1`across the board wherever they use Btrfs, no exceptions. Example 2: - Installer sets mount option `-o compress=zstd:1` - Installer sets `NOCOMPRESS` on /home - user enables it per directory or file using an xattr, via `btrfs property set` or exposed by desktop integration in the GUI A call for testing could be useful in helping to make decisions, and provide some "how to" documentation to encourage it. The generic issue and technical concerns of actually implementing this across Fedora, are in [fedora-btrfs / project: enable compression on btrfs installs](https://pagure.io/fedora-btrfs/project/issue/5) This issue for Workstation Edition, to discuss UI/UX and integration related issues.
catanzaro commented 2020-10-28 14:57:48 +00:00
Owner
Copy link

Proposal: default to compress=zstd:1 for F34

Proposal: default to `compress=zstd:1` for F34
chrismurphy commented 2020-11-03 15:02:33 +00:00
Author
Owner
Copy link

Metadata Update from @chrismurphy:

  • Issue assigned to chrismurphy
**Metadata Update from @chrismurphy**: - Issue assigned to chrismurphy
chrismurphy commented 2020-12-15 14:19:16 +00:00
Author
Owner
Copy link

Metadata Update from @chrismurphy:

  • Issue tagged with: meeting-request
**Metadata Update from @chrismurphy**: - Issue tagged with: meeting-request
catanzaro commented 2021-01-02 22:32:40 +00:00
Owner
Copy link

Metadata Update from @catanzaro:

  • Issue untagged with: meeting-request
  • Issue tagged with: meeting
**Metadata Update from @catanzaro**: - Issue **un**tagged with: meeting-request - Issue tagged with: meeting
catanzaro commented 2021-01-12 00:21:20 +00:00
Owner
Copy link

WG is happy with this proposal. It is pending FESCo approval in https://pagure.io/fesco/issue/2538.

WG is happy with this proposal. It is pending FESCo approval in https://pagure.io/fesco/issue/2538.
catanzaro commented 2021-01-12 00:21:21 +00:00
Owner
Copy link

Metadata Update from @catanzaro:

  • Issue untagged with: meeting
**Metadata Update from @catanzaro**: - Issue **un**tagged with: meeting
catanzaro commented 2021-02-02 16:24:45 +00:00
Owner
Copy link

This change is approved by FESCo. Was it successfully implemented yet?

This change is approved by FESCo. Was it successfully implemented yet?
ngompa commented 2021-02-02 16:32:25 +00:00
Owner
Copy link

@salimma Any progress?

@salimma Any progress?
salimma commented 2021-02-02 16:58:52 +00:00
Owner
Copy link

Working on it this week, sorry for the late update!

Working on it this week, sorry for the late update!
chrismurphy commented 2021-02-02 17:22:28 +00:00
Author
Owner
Copy link

I guess the easy part is adding this option to the autopart line in every desktop kickstart file? From my reading of the doc and prior conversation with Anaconda folks, is that this will use the specified mount option for the installation as well as adding it to /etc/fstab.

--fsoptions compress=zstd:1

Harder is figuring out which kickstart files to modify. I think the following three?

$ grep autopart *.ks
fedora-cloud-base.ks:autopart --noboot --nohome --noswap --nolvm
fedora-container-common.ks:autopart --noboot --nohome --noswap --nolvm
fedora-disk-minimal.ks:autopart --type=plain --noswap
fedora-disk-server.ks:autopart --noswap
fedora-disk-workstation.ks:autopart --type=plain --noswap
fedora-disk-xfce.ks:autopart --type=plain --noswap
fedora-iot.ks:autopart --nohome --noswap --type=plain
fedora-modular-disk-minimal.ks:autopart --type=plain

(It's a bit off topic but I'm confused about --type=plain)

And hardest is how to change it with the two ARM desktop images.

  • compress=zstd:1 at image creation time and fstab; prevents the subsequent xz operation from doing much of anything, resulting in a much larger image size.
  • compress=zstd:15 at image creation time but zstd:1 in fstab; it'll still be a bit bigger than the xz image we have now but then the xz step (and also the user unxz step) can be skipped. Possibly zstd:11 is the best bang for the buck, small detail.
  • compress=zstd:1 only to the /etc/fstab; the resulting installation starts with uncompressed Btrfs, but all subsequent writes will be compressed, and over time most everything will become compressed.

No matter which of the three, if it's possible to add an fstrim before the final umount, there will be a net reduction in the image. Right now, there's no fstrim for either ARM image, and there's a bunch of stale deleted packages left in the image. It's really suboptimal.

I guess the easy part is adding this option to the `autopart` line in every desktop kickstart file? From my reading of the [doc](https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html) and prior conversation with Anaconda folks, is that this will use the specified mount option for the installation as well as adding it to /etc/fstab. `--fsoptions compress=zstd:1` Harder is figuring out which kickstart files to modify. I think the following three? $ grep autopart *.ks ~~fedora-cloud-base.ks:autopart --noboot --nohome --noswap --nolvm~~ ~~fedora-container-common.ks:autopart --noboot --nohome --noswap --nolvm~~ fedora-disk-minimal.ks:autopart --type=plain --noswap ~~fedora-disk-server.ks:autopart --noswap~~ fedora-disk-workstation.ks:autopart --type=plain --noswap fedora-disk-xfce.ks:autopart --type=plain --noswap ~~fedora-iot.ks:autopart --nohome --noswap --type=plain~~ ~~fedora-modular-disk-minimal.ks:autopart --type=plain~~ (It's a bit off topic but I'm confused about `--type=plain`) And hardest is how to change it with the two ARM desktop images. * compress=zstd:1 at image creation time and fstab; prevents the subsequent xz operation from doing much of anything, resulting in a much larger image size. * compress=zstd:15 at image creation time but zstd:1 in fstab; it'll still be a bit bigger than the xz image we have now but then the xz step (and also the user unxz step) can be skipped. Possibly zstd:11 is the best bang for the buck, small detail. * compress=zstd:1 only to the /etc/fstab; the resulting installation starts with uncompressed Btrfs, but all subsequent writes will be compressed, and over time most everything will become compressed. No matter which of the three, if it's possible to add an `fstrim` before the final umount, there will be a net reduction in the image. Right now, there's no fstrim for either ARM image, and there's a bunch of stale deleted packages left in the image. It's really suboptimal.
ngompa commented 2021-05-03 16:43:21 +00:00
Owner
Copy link

This is done now, so I'm closing this.

This is done now, so I'm closing this.
ngompa commented 2021-05-03 16:43:21 +00:00
Owner
Copy link

Metadata Update from @ngompa:

  • Issue close_status updated to: Fixed
  • Issue status updated to: Closed (was: Open)
**Metadata Update from @ngompa**: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
No results found.
Labels
Clear labels   
a11y

Issues related to Fedora GNOME accessibility

  
btrfs
  
Closed As
Can't fix
Closed with the reason of Can't fix

  
Closed As
Deferred to upstream
Closed with the reason of Deferred to upstream

  
Closed As
Fixed
Closed with the reason of Fixed

  
Closed As
Won't fix
Closed with the reason of Won't fix

  
default-apps

   
easyfix

Easy fixes for new contributors

  
experience

Issues related to the desktop experience

  
help-wanted

Volunteers wanted to help with this issue

  
installation

Issues related to the installer

  
meeting

For next Workstation working group meeting

  
meeting-request

Propose issue for a future meeting

  
nvidia

   
packaging

Requires packaging work

  
pending-action

Has an action that hasn't been completed

  
qa

Related to QA processes, infrastructure, policy. (Not for any issue which affects product quality.)

  
testing

Requires testing and validation

  
user-docs

Issues with the workstation user docs

  
wg-docs

Issues with the working group docs

  
wg-meta

For Workstation WG issues

No labels
a11y
btrfs
Closed As
Can't fix
Closed As
Deferred to upstream
Closed As
Fixed
Closed As
Won't fix
default-apps
easyfix
experience
help-wanted
installation
meeting
meeting-request
nvidia
packaging
pending-action
qa
testing
user-docs
wg-docs
wg-meta
Milestone
Clear milestone
No items
No milestone
Fedora 34
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
workstation/tickets#177
No description provided.