Migration from SVN to Bitbucket fails for large number of revisions

SubGit
#1

Hi. We are using subgit for migration of SVN repositories to Bitbucket. The automated migration fails for repositories having large number of revisions. Is there a config file or some way we can have these migrations complete successfully.

#2

Hello Naseem,

There is no common config file, but the configuration can be created manually based on the particular SVN project layout. However, the issue should be investigated because if the auto-configuration phase fails, for example, due to errors in the SVN repository, then the migration will fail, too. So I suggest to find out why the faults occur and resolve it. We would need logs and a little more details on the problems, namely:

  • which tool is in use – SubGit command-line tool or you’re talking about SVN Mirror add-on for Bitbucket server?
  • if you’re using SubGit which exact commands were used for the configuration and migration?
  • what happens to the automated migration – which errors appear or what happens during the migration.

SubGit stores logs right in the repository, in ‘subgit/logs’ directory. The SVN Mirror add-on logs can be collected on the Repository settings - SVN Mirror - Support tab.

#3

This was setup by someone who no longer works with our organization. I will try to provide all the info you are looking for.

We are using a Jenkins build to migrate the data from SVN to Bitbucket. From the console output of the build I see we are using subgit

  • Configuring SubGit *

[MISC_SVN2Git_Migration] $ /bin/sh -xe /opt/jenkins/tmp/jenkins82689424833269048.sh

  • /opt/jenkins/jenkins-tools/subgit/3.2.4/bin/subgit --version
    SubGit version 3.2.4 (‘Bobique’) build #3670
    © TMate Software 2012-2017 (http://subgit.com/)
  • /opt/jenkins/jenkins-tools/subgit/3.2.4/bin/subgit configure --svn-url https://svn.gm.com/svn/SS_AOM_49101 from-svn
    SubGit version 3.2.4 (‘Bobique’) build #3670

Configuring writable Git mirror of remote Subversion repository:
Subversion repository URL : https://svn.gm.com/svn/SS_AOM_49101
Git repository location : from-svn

This is the error I got,

INSTALLATION FAILED

error: svn: E175002: SSL peer shut down incorrectly
error: svn: E175002: REPORT request failed on ‘/svn/SS_AOM_49101/!svn/vcc/default’
error: SSL peer shut down incorrectly
error: Unexpected error has occurred; please report along with the logs (’/opt/jenkins/workspace/MISC_SVN2Git_Migration/subgit-install-20190410-101056.zip’)
error: to http://issues.tmatesoft.com/, thank you!
Build step ‘Execute shell’ marked build as failure
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: naseem.banu@gm.com
Finished: FAILURE

#4

Hello Naseem

first of all, I’d like to note that version 3.2.4 is very old, many errors have been fixed since that version so I’d recommend updating SubGit to the latest version of 3.3.6 as the first step.
The ‘E175002: SSL peer shut down incorrectly’ appears to happen on SVN side, SSL connection was cut off in the middle of pulling and indexing, so it would worth to check the SVN server, too, if it is able to be accessed over https from other machines.
If the SVN server can be accessed from other machines but SubGit 3.3.6 has problems connecting it anyway, please provide the installation logs (the ‘subgit-install’ zip) for analysis.

#5

Yes i will look at installing the new version. Yes, SVN server can be accessed from other machines. Another thing to note is the migration is n’t failing for small/medium number of revisions. It only fails for large repositories. So do you think it is an connection issue?

#6

Yes, it could be a connection issue, as SubGit needs some time to fetch bit repositories/revisions and the SVN may close the connection unexpectedly for SubGit during the server response processing. A possible workaround is to set ‘svn.httpSpooling’ setting to ‘true’ in the SubGit configuration file – this setting tells SubGit to spool server response to disk instead of processing it directly, that may help in case of timeouts. Note, though, in this case, SubGit can take much disk space, depending on what comes from SVN.
Yet I’d recommend first to update SubGit as there are fixes for files handling and connections issues, too.

#7

I tried checking all the files under subgit installation. Nowhere i could find svn.httpSpooling property. What is the path of the configuration file you talked about.

#8

Hello Naseem
I just found out that your question had been left unanswered, I apologize for that inconvenience.
By the ‘svn.httpSpooling’ setting I meant the httpSpooling option in [svn] section in the SubGit configuration file, it can be set right in the /subgit/config file:

[svn]
…
url = 
…
trunk = …
…
httpSpooling = true