Speed RagTag up a few orders of magnitude on fragmented assemblies with caching

[twrightsman]

I noticed computing the line numbers of AGP files and objects was ~99% (!) of the runtime on our very fragmented contig assemblies.

This patch pre-computes the line numbers and dropped runtime from 16 hours to 8 minutes, the vast majority of which was spent computing the AGP file to write out.

I have checked this produces identical output on our data, but probably needs more extensive testing.

[mschatz]

This is amazing! @mike would you be able to review and integrate? Thanks so much for your contribution! Mike

…

On Wed, Feb 14, 2024 at 9:51 AM Travis Wrightsman ***@***.***> wrote: I noticed computing the line numbers of AGP files and objects was ~99% (!) of the runtime on our very fragmented contig assemblies. This patch pre-computes the line numbers and dropped runtime from *16 hours to 8 minutes*, the vast majority of which was spent computing the AGP file to write out. I have checked this produces identical output on our data, but probably needs more extensive testing. ------------------------------ You can view, comment on, or merge this pull request online at: #178 Commit Summary - 4b43afa <4b43afa> Cache AGP file and object line counts File Changes (1 file <https://github.com/malonge/RagTag/pull/178/files>) - *M* ragtag_utilities/AGPFile.py <https://github.com/malonge/RagTag/pull/178/files#diff-52e0b172693fb269551599232c875d33a2a928e468d3846a65083dd0d552febb> (16) Patch Links: - https://github.com/malonge/RagTag/pull/178.patch - https://github.com/malonge/RagTag/pull/178.diff — Reply to this email directly, view it on GitHub <#178>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABP347QJFQLRNG2S2SZWNLYTTFOTAVCNFSM6AAAAABDINRPG2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZTINJSGY3DONI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>