Resuming/Restarting a Partially-Generated Bag

[bishbashbackup]

Hiya,

I'm wanting to find out if there's currently a way to restart an interrupted bagit process, without losing all progress. Had a look on previous issues raised, but couldn't find any mention of this.

For context, we're wanting to use bagit.py for checksumming files that are getting transferred to the archive where I work. However, the deposits we receive can contain many TBs of data. We've had experiences where we've tried to run bagit on these packages overnight, only to find in the morning that the process got tripped up by an unforeseen issue (i.e. network dropout, file permission, etc.), ending up with a partially-generated bag.

As a result, I'm wondering if there is any way to resume these partially-generated bags. Whilst, this may be unnecessary for smaller bags, for larger bags this would save a significant amount of processing time and make the process smoother.

Thanks in advance,
Alex

[kieranjol]

Hi Alex, I think would be very useful, also for validating large bags! It would require a different method of bagit-python working however - it would need to move away from just working in memory and only writing manifests etc when the processes are fully completed. It would require some sort of temp files (either a renamed version of the actual manifests which are renamed to manifest-NNN.txt at the end of the job, or tempfiles existing in another location) that are steadily writing each manifest line, and then perhaps waiting until the very end to write the remaining tag-manifests and bag-info etc so that it's clear that without these present, the process is not actually finished. There's probably many more considerations on top of all that, however I do think it could be worth it. Best, Kieran O’Leary, National Library of Ireland

…

On Mon 23 Mar 2026 at 11:10, Alex Habgood ***@***.***> wrote: *bishbashbackup* created an issue (LibraryOfCongress/bagit-python#201) <#201> Hiya, I'm wanting to find out if there's currently a way to restart an interrupted bagit process, without losing all progress. Had a look on previous issues raised, but couldn't find any mention of this. For context, we're wanting to use bagit.py for checksumming files that are getting transferred to the archive where I work. However, the deposits we receive can contain many TBs of data. We've had experiences where we've tried to run bagit on these packages overnight, only to find in the morning that the process got tripped up by an unforeseen issue (i.e. network dropout, file permission, etc.), ending up with a partially-generated bag. As a result, I'm wondering if there is any way to resume these partially-generated bags. Whilst, this may be unnecessary for smaller bags, for larger bags this would save a significant amount of processing time and make the process smoother. Thanks in advance, Alex — Reply to this email directly, view it on GitHub <#201?email_source=notifications&email_token=AAITFPQOSWRBWNJAQ6OXBO34SELRXA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NBRGIYDANRSGE3TBJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAITFPXE2O5UHT7ZLKP4G4L4SELRXAVCNFSM6AAAAACW36BADCVHI2DSMVQWIX3LMV43ASLTON2WKOZUGEZDAMBWGIYTOMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>