infra/tickets

Fork 1

discourse s3 backup buckets no longer active #11683

New issue

Open

opened 2023-12-15 17:05:42 +00:00 by mattdm · 12 comments

mattdm commented

2023-12-15 17:05:42 +00:00

The S3-bucket-sync backups we have for Fedora Discussion are an exception Discourse's normal process, and as part of renewing our hosting contract this year, we've agreed to no longer require them.

The only thing missing from the regular backups is image and other file uploads. I am working on a script which will archive these via the API. In the meantime, the S3 sync is not happening.

Once this is ready, I'll want somewhere to run that (and to store the results). For now, there is no action needed, but be aware that these buckets are no longer updated.

The S3-bucket-sync backups we have for Fedora Discussion are an exception Discourse's normal process, and as part of renewing our hosting contract this year, we've agreed to no longer require them. The only thing missing from the regular backups is image and other file uploads. I am working on a script which will archive these via the API. In the meantime, the S3 sync is not happening. Once this is ready, I'll want somewhere to run that (and to store the results). For now, there is no action needed, but be aware that these buckets are no longer updated.

phsmoura commented

2023-12-18 19:11:54 +00:00

Member

Metadata Update from @phsmoura:

Issue tagged with: aws, low-gain, low-trouble, ops

**Metadata Update from @phsmoura**: - Issue tagged with: aws, low-gain, low-trouble, ops

phsmoura commented

2023-12-20 19:00:33 +00:00

Member

Metadata Update from @phsmoura:

Issue priority set to: Waiting on Assignee (was: Needs Review)

**Metadata Update from @phsmoura**: - Issue priority set to: Waiting on Assignee (was: Needs Review)

kevin commented

2024-01-08 21:54:41 +00:00

Owner

Out of curiosity, what is the 'regular backups' ?

zlopez commented

2024-03-04 14:38:36 +00:00

Owner

Metadata Update from @zlopez:

Issue priority set to: Waiting on Reporter (was: Waiting on Assignee)

**Metadata Update from @zlopez**: - Issue priority set to: Waiting on Reporter (was: Waiting on Assignee)

mattdm commented

2024-03-13 20:24:21 +00:00

Author

There is a timed job on the host itself that makes backup archives, and we can get these with the API.

But, yeah.

When I get a chance I will prototype this in Communishift.

There is a timed job on the host itself that makes backup archives, and we can get these with the API. But, yeah. When I get a chance I will prototype this in Communishift.

zlopez commented

2024-10-10 16:31:32 +00:00

Owner

[backlog refinement]
Any progress on this @mattdm?

[backlog refinement] Any progress on this @mattdm?

kevin commented

2025-02-20 17:43:41 +00:00

Owner

[backlog refinement]
Any news here? Or perhaps this is something that someone else could take on (with initial info from you)?

@mattdm

[backlog refinement] Any news here? Or perhaps this is something that someone else could take on (with initial info from you)? @mattdm

kevin commented

2025-11-06 17:45:09 +00:00

Owner

I'm not sure if this is still needed? Does discourse do backups with uploads now?

kevin commented

2026-03-11 23:45:48 +00:00

Owner

The current help says: "Discourse backups include the full site database, which contains everything on the site: topics, posts, users, groups, settings, themes, etc. Depending on how the backup file is created, it may or may not include uploads. "

I can't seem to see the backup job, it might be at a different level...

kevin commented

2026-06-04 15:39:12 +00:00

Owner

I'll try and catch @mattdm and ask about this and clarify what the status is.

kevin self-assigned this

2026-06-04 15:39:19 +00:00

kevin added

and removed

labels

2026-06-04 15:40:46 +00:00

mattdm commented

2026-06-04 18:28:23 +00:00

Author

The current backups do not contain uploads/images. The linked upstream topic says "Warning: If your site is hosted by us and you would like to create a backup that includes uploads, contact the Discourse team before creating the backup."

I never did get very far on the promised S3-download script for alternate backups. I think it's probably still a good idea.

The current backups _do not_ contain uploads/images. The linked upstream topic says "Warning: If your site is hosted by us and you would like to create a backup that includes uploads, contact the Discourse team before creating the backup." I never did get very far on the promised S3-download script for alternate backups. I think it's probably still a good idea.

kevin commented

2026-06-09 19:18:37 +00:00

Owner

ok. I talked with our discourse contacts.

Currently:

They do backups every 12 hours
These backups DO include uploads
They keep 30 days of them in case of disaster recovery needed

If we want backups of our own, we can do that, but those don't have uploads included.
So, I guess we still need a way to download those uploads, or asking for disaster recovery backup copy if we desire to have uploads backed up ourselves instead of relying on them for them.

ok. I talked with our discourse contacts. Currently: * They do backups every 12 hours * These backups DO include uploads * They keep 30 days of them in case of disaster recovery needed If we want backups of our own, we can do that, but those don't have uploads included. So, I guess we still need a way to download those uploads, or asking for disaster recovery backup copy if we desire to have uploads backed up ourselves instead of relying on them for them.