User directories as Btrfs subvolumes by default #232

Open
opened 2021-06-15 18:37:33 +00:00 by ngompa · 23 comments
Owner

shadow-utils since version 4.7 has supported the ability to create user home directories as subvolumes.

As an iterative step toward supporting use-cases for isolating and protecting each user's data individually (to lay the groundwork for disaster recovery tooling as well as encryption as tracked in #82), it may be a good idea to start defaulting to creating subvolumes on creating new users when on Btrfs.

`shadow-utils` since version 4.7 has supported [the ability to create user home directories as subvolumes](https://github.com/shadow-maint/shadow/commit/c1d36a8acb1d5fbe42099a37c8ed6f33e8a648e6). As an iterative step toward supporting use-cases for isolating and protecting each user's data individually (to lay the groundwork for disaster recovery tooling as well as encryption as tracked in #82), it may be a good idea to start defaulting to creating subvolumes on creating new users when on Btrfs.
Author
Owner

General Fedora Btrfs ticket is tracked as fedora-btrfs/project#50.

General Fedora Btrfs ticket is tracked as fedora-btrfs/project#50.
Owner

This seems like an aspect of the discussion about homed and user data encryption. Do we need a separate ticket for it?

This seems like an aspect of the discussion about homed and user data encryption. Do we need a separate ticket for it?
Author
Owner

This is orthogonal to homed, since we can do this without homed today.

This is orthogonal to homed, since we can do this without homed today.
Owner

This is orthogonal to homed, since we can do this without homed today.

But it seems like it would be better to wait and see how homed plays out? Or, if you think we should do it ASAP, then let's put that to the WG to decide.

> This is orthogonal to homed, since we can do this without homed today. But it seems like it would be better to wait and see how homed plays out? Or, if you think we should do it ASAP, then let's put that to the WG to decide.
Author
Owner

The main value for doing it now is making it easier to something like Time Machine for user data, since snapshotting or doing btrfs send of a subvolume is possible.

The main value for doing it now is making it easier to something like Time Machine for user data, since snapshotting or doing `btrfs send` of a subvolume is possible.
Owner

Maybe some folks could try this layout locally, and see what happens? What are the cons? fedora-btrfs/project#50 has a couple of concerns listed: backup programs and the installer.

@lennart told us a few weeks ago that there's an early idea (perhaps it's "could happen" more than "will happen") of having systemd-homed learn about Btrfs snapshotting for this purpose; it already has subvolume support: A "btrfs" subvolume for each user, also located in /home/.homedir. This provides no encryption, but good quota support.*

Is the idea to have a way to convert from one subvolume regime to another? Just because it's subvolume based doesn't mean it'll "just work" with the systemd-homed way of doing things. Do we really want to do that work? We'd have to plan for that. Or alternative just wait and have one change instead of, in-effect, two changes? And while it's fairly simple to convert from one subvol regime to another (cp -r) it's harder to convert from one snapshot regime to another. We don't have a spec for any of that, it's all very tool specific with a ton of assumptions made by each tool.

And so on...

Maybe some folks could try this layout locally, and see what happens? What are the cons? fedora-btrfs/project#50 has a couple of concerns listed: backup programs and the installer. @lennart told us a few weeks ago that there's an early idea (perhaps it's "could happen" more than "will happen") of having systemd-homed learn about Btrfs snapshotting for this purpose; it already has subvolume support: *A "btrfs" subvolume for each user, also located in /home/*.homedir. This provides no encryption, but good quota support.* Is the idea to have a way to convert from one subvolume regime to another? Just because it's subvolume based doesn't mean it'll "just work" with the systemd-homed way of doing things. Do we really want to do that work? We'd have to plan for that. Or alternative just wait and have one change instead of, in-effect, two changes? And while it's fairly simple to convert from one subvol regime to another (cp -r) it's harder to convert from one snapshot regime to another. We don't have a spec for any of that, it's all very tool specific with a ton of assumptions made by each tool. And so on...
Owner

Seems like this would benefit from a wider discussion - setting the meeting label.

Seems like this would benefit from a wider discussion - setting the meeting label.
Owner

Metadata Update from @aday:

  • Issue tagged with: meeting
**Metadata Update from @aday**: - Issue tagged with: meeting
Author
Owner

From last week's meeting:

ACTION: Neal to file the bugs he found and get them fixed. Then we can revisit this. Chris to help out too (cmurf, 03:46:54)

From last week's meeting: > ACTION: Neal to file the bugs he found and get them fixed. Then we can revisit this. Chris to help out too (cmurf, 03:46:54)
Author
Owner

Metadata Update from @ngompa:

  • Issue untagged with: meeting
  • Issue assigned to ngompa
  • Issue tagged with: pending-action
**Metadata Update from @ngompa**: - Issue **un**tagged with: meeting - Issue assigned to ngompa - Issue tagged with: pending-action
Author
Owner

Refreshing my memory on this:

  • shadow-utils set the home subvolume to be owned by root instead of the new user, need to verify if this is still a problem. If it is, file a bug.
Refreshing my memory on this: * `shadow-utils` set the home subvolume to be owned by `root` instead of the new user, need to verify if this is still a problem. If it is, file a bug.
Owner

@ngompa looks like you have the action item here.

@ngompa looks like you have the action item here.
Owner

@ngompa could you provide a status update please?

@ngompa could you provide a status update please?
Owner

We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption.

This change is still wanted. The next step is to figure out how to use shadow-utils to do it - if someone wants to help with that, then that would be great.

We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption. This change is still wanted. The next step is to figure out how to use shadow-utils to do it - if someone wants to help with that, then that would be great.
Owner

Metadata Update from @aday:

  • Issue tagged with: help-wanted
**Metadata Update from @aday**: - Issue tagged with: help-wanted
Owner
[This might break the trash feature?](https://gitlab.gnome.org/GNOME/glib/-/issues/1885)
Owner

We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption.

This change is still wanted. The next step is to figure out how to use shadow-utils to do it - if someone wants to help with that, then that would be great.

@ngompa can you provide a status update on this please? The action item is assigned to you.

> We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption. > > This change is still wanted. The next step is to figure out how to use shadow-utils to do it - if someone wants to help with that, then that would be great. @ngompa can you provide a status update on this please? The action item is assigned to you.
Author
Owner

It fell off my radar, but it's on my list to investigate and test, especially as a precursor to other interesting work around encryption and easy replication.

It fell off my radar, but it's on my list to investigate and test, especially as a precursor to other interesting work around encryption and easy replication.
Owner

We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption.

This doesn't seem like a real advantage to me. When backing up, you surely want to exclude locations like cache directories and trash, and possibly git repos, and possibly container images and virtual machines, etc. Certainly you don't want to back up caches. In practice, my home directory often has 500 GB or more of data that should not be backed up and only about 30 GB of data that does belong in backups.

Looking over the concerns raised in https://pagure.io/fedora-btrfs/project/issue/50, it actually looks like this change would break several popular backup programs. Maybe we should just not do it?

> We discussed this on Tuesday's working group call. It seems that the main advantage of user directories as subvolumes is that, when encrypted, they can be backed up without the need for decryption. This doesn't seem like a real advantage to me. When backing up, you surely want to exclude locations like cache directories and trash, and possibly git repos, and possibly container images and virtual machines, etc. Certainly you don't want to back up caches. In practice, my home directory often has 500 GB or more of data that should not be backed up and only about 30 GB of data that does belong in backups. Looking over the concerns raised in https://pagure.io/fedora-btrfs/project/issue/50, it actually looks like this change would break several popular backup programs. Maybe we should just not do it?
Owner

@ngompa are you sure we still want to do this?

@ngompa are you sure we still want to do this?
Author
Owner

We do, but there's currently no way to adjust shadow to default to this. A ticket is open with upstream shadow about this.

We do, but there's currently no way to adjust shadow to default to this. A [ticket is open with upstream shadow](https://github.com/shadow-maint/shadow/issues/1162) about this.
Owner

Looks like we have two concerns here:

This might break the trash feature?

We'll just have to fix GLib.

Looking over the concerns raised in https://pagure.io/fedora-btrfs/project/issue/50, it actually looks like this change would break several popular backup programs. Maybe we should just not do it?

We're OK with doing this anyway, and expect backup programs to fix themselves?

We do, but there's currently no way to adjust shadow to default to this. A ticket is open with upstream shadow about this.

This seems manageable.

Looks like we have two concerns here: > [This might break the trash feature?](https://gitlab.gnome.org/GNOME/glib/-/issues/1885) We'll just have to fix GLib. > Looking over the concerns raised in https://pagure.io/fedora-btrfs/project/issue/50, it actually looks like this change would break several popular backup programs. Maybe we should just not do it? We're OK with doing this anyway, and expect backup programs to fix themselves? > We do, but there's currently no way to adjust shadow to default to this. A [ticket is open with upstream shadow](https://github.com/shadow-maint/shadow/issues/1162) about this. This seems manageable.
Owner

Metadata Update from @catanzaro:

  • Issue untagged with: pending-action
**Metadata Update from @catanzaro**: - Issue **un**tagged with: pending-action
Sign in to join this conversation.
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
workstation/tickets#232
No description provided.