We have recently migrated a repository from SVN to bitbucket but are not able to import the repository to any other location.
Also, we are seeing below errors when trying to import the migrated repository-
2020-09-18 12:59:39,680 ERROR [pool-8-thread-7] c.a.b.i.i.repository.RefSyncTask Failed to import repository: NQ/clock-work-tools2[34678]
com.atlassian.bitbucket.scm.CommandFailedException: ‘/opt/app/dts/git/v2.9.5/bin/git -c gc.auto=0 -c credential.username=kumaspr -c credential.helper= fetch --force --prune --progress http://kumaspr@vstdtsp03f.server.rbsgrp.mde:7091/scm/svnbit/clock-work-tools2.git +refs/:refs/’ exited with code 128 saying: remote: fatal: Out of memory, malloc failed (tried to allocate 527200769 bytes)
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onError(DefaultCommandExitHandler.java:47)
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:111)
at com.atlassian.bitb
Below is the configuration that we used for migration-
SVN Url -https://svn.dts.fm.rbsgrp.net/clockwork-tools
Mappings -
trunk = :refs/heads/master
Is it possible that the repository got corrupted during the migration?
Also, attaching the SVN mirror logs for reference.
svnmirror_support_r29032_20200921_135233_175.zip (279.7 KB)
Hello
I don’t see any signs in logs that anything wrong happened during the import, the import logs looks pretty good. And judging from the error message you shared, this is not about repository corruption, it’s caused by that git is unable to allocate enough memory to perform the requested “fetch” operation. This can be resolved by adjusting Git configuration as it’s described here, for example:
It’s also may make sense to upgrade Git to latest version if it’s too old.
Hi,
Thank you for your inputs. There is one more thing that we are observing with this migrated repository that when we migrated it, the size of the repository appeared to be approx 500 MB but now it is showing 0 bytes.
If we go and open any file in source code and check the size again for the repo, it again starts showing up the correct repository size.
We haven’t seen such issue with many hundreds of other migrations that we did and upgrading the git version will be a time taking process on our side.
Would there be any reason that this specific repository is running into memory issues?
Thanks
Hi,
it’s not completely clear where do you see the repository size and how is it being measured. Taking into account this size displaying changes after a file is opened in the repository, it sounds more like a UI issue (like the UI is not being updated correctly) rather than actual data issue.
I haven’t found any issues in SVN Mirror add-on logs, the import process looks good in logs and finished successfully. However, it was only imported once and I don’t know what was happening to the repository since then – the add-on logs don’t contain such information as the repository only was imported once and the add-on was not working in the repository anymore. So it’s hard to say what is the current repository state, this should be checked with ‘git fsck’ or similar tools.
The most probably reason for the Git out of memory issue is that there are too big files in the repository that Git cannot handle. Git upgrade is not a requirement, better to stick with the Git settings approach.