Batch sync SVN repos to Bitbucket

SVN Mirror
#1

Hello Is there a way to run the SVN mirror in “batch” to get a shot of all our SVN repos to a single Bitbucket project? We currently have one project with 65+ repos. Being able to run a batch job can save our users a lot of time.

#2

Hello Peter!
Do you mean you want to mirror 65 your SVN repositories into 65 different Git repositories within one project?

Yes, there’s a way: use SVN Mirror add-on REST API. See Section 2 of the following article:

https://subgit.com/documentation/stash-rest-api.html#how-to-use-rest-api-to-configure-gitsvn-mirror-repository

If you create a script like described there and fill in the translation settings for each repository (the settings that you otherwise would specify manually) you can create 65 mirrors in a batch.

Some details to pay attention to.

  1. I would recommend the URL ending with
    {code}
    …/configure?start=mirror&async=true
    {code}

(Edit: I’ve noticed a typo in the documentation: confiugre -> configure; my colleague will fix it today).

This means that mirror will start after initial translation and that the command will exit immediately, just putting the task to the task queue instead of waiting for translation completion before starting processing the next SVN URL.

  1. Make sure you have correct
    {code}
    “layout” : {
    “type” : “AUTO|DIRECTORY|MANUAL”,

    options for AUTO layout type (optional):

    “trunkPath” : “path/to/trunk/directory”,

    options for MANUAL layout type (optional):

    “trunk” : “trunk:refs/heads/master”,
    “branches” : [“branches/:refs/heads/”, …],
    “tags” : [“tags/:refs/tags/”, …],
    “shelves” : [“shelves/:refs/shelves/”, …],
    “excludePaths” : [],
    “excludeBranches” : []
    },
    {code}

section. It is critical to specify correct trunk/branches/tags/shelves options depending on repository structure. You can find more information in https://subgit.com/documentation/branches-mapping.html article.

This section is analog of these options: https://subgit.com/documentation/bitbucket.html#mirror_settings

If you have any questions regarding it, feel free to ASK.

  1. Make sure you have correct PROJECT_KEY and REPO_SLUG. Note: you don’t have to create the repositories manually, SVN Mirror add-on will create the repositories automatically if they are absent.

Also important: REPO_SLUG should be in lowercase while PROJECT_KEY should be in uppercase. E.g.
{code}
http://…/projects/TEST/repos/test/…
{code}

I hope this information helps. Feel free to contact us again if you have any questions.

#3

I would also like to add that it’s convenient to see and control the list of all repositories on SVN Mirror settings for Project (not particular repository) OR even on SVN Mirror add-on global settings page: Administration | SVN Mirror | Mirror Repositories.

Also on this page on tab “Add-on Settings” in “Performance” section you can set the number of threads in the pool. By default this value is 8, this means that 8 repositories at a time will be translated while all the others will wait until there’re free threads. Depending on your resources and performance requirements you might want to choose different values.

#4

How does the REST API know to loop through all the 65 repos without specifically naming each repo? Can we define the repo “family” that contains all our 65 SVN repos?

#5

REST API doesn’t know anything about looping. What I had on my mind is that you could create a bash script looping over urls:

{code}
for url in cat /tmp/list-of-urls ; do

call ‘curl’ command on a corresponding $url with corresponding settings

done
{code}

So the task of looping over URLs lies outside SVN Mirror add-on, it doesn’t have any means of working with lists of URLs, only individual URLs.