When importing one of our subversion repositories, the subgit install operation will run forever. It generates a large log file in the process where it seems to be repeating operations in a loop.
Please advise.
Hello David,
thank you for reaching out to us on that matter.
May I ask you how long the initial import runs by now and how big the SVN repository is (how many revisions in the repository)? What has made you to believe the operations are running in a loop – I mean, what are the patterns you see in the logs that you treat as a loop? Also, could you please share the log files with us for analysis?
Ildar,
The initialization step for this particular repository occurs relatively quickly, I think less than then a minute of time.
This repository is for a embedded hardware project. It has a lot of SVN extern references in it, I think it may have something to do with it, but i’m not sure. I’ve read through the documentation about how subgit handles extern references for this process, but I’m not sure if that’s what’s affecting this process.
Statistics: 374 commits, with a total of 129,048 file changes have been registered by this repository.
I see quite a bit of repetition in the log files. I let the install process run for a full 10 minutes and it didn’t complete, Full log dump is over 72 MBs from that attempt.
I’m unable to upload a reduced log file. The limit in your dialog says 50MB, but it fails on 11MB, so i’m not sure why that isn’t working.
I will attach logs in another response.
I was able to compress the log using 7Zip and upload it to our google drive. The following link should give you access to the logs.
https://drive.google.com/file/d/1crcf7wQkQ-q_pHnJbVli1RorVkslBLK9/view?usp=sharing
Hello David,
thank you for the log.
As far as I see in the log, SubGit found 9479 revisions in SVN repository and started the import. The mapping configuration supposes standard SVN layout, which, as it appears, does not reflect the actual SVN repository layout well enough, so the actual import started from the revision 5973 and, judging from the log, SubGit is busy getting data for this revision. I haven’t found any repetitions in logs, there are plenty of event like
successfully received 'trunk/…' with size=30723
but that is pretty normal, SubGit just reports the file it currently handles.
As for the time of the translation – it is pretty normal that the import lasts longer than 10 minutes, for more or less old repositories it’s not unusual to have hours or even days for the initial import, just because SubGit needs time to handle the SVN history and get all the data from SVN side which is not a momentarily process. I don’t see any signs in the log that anything might be wrong, SubGit just is doing its work to get the data from SVN.
As for SVN externals configured in the SVN repository – I’m afraid, SubGit does not support externals and thus it won’t import them into Git, only the main repository data will be imported. The only thing SubGit is able to do with SVN externals is to translate their configuration to special .gitsvnextmodules files that would contain externals definitions. It’s not being performed by default, to enable this feature the following setting should be added to the configuration file:
[translate]
externals = true
This setting should be added prior to the initial import start to have the files created for all the externals.
I must to note here that Git has no special features to handle those files automatically; SmartGit client had some functionality allowing to create submodules out of those file, but I’m not sure it still support this feature in the recent versions. A possible workaround in case if you need those external repositories to be embedded in the main one on the Git side is to translate those repositories to Git individually and then add them to the main Git repository as submodules or subtrees.
Also, it may be interesting for you, we are developing a new program that would allow embedding repositories on the server side, here are the details:
Please let us know if you are interested, we will be glad to discuss it in details.
OK. Thank you for the information Ildar. This import ran very long compared to all of the other repositories. I will re-run it, and let it continue to run.
We are relatively new to git. I took a look at the link you provided, and it looks interesting. We are currently hosting a gitlab instance, and it appears that your product isn’t ready for that yet.
Any additional information you could provide on the process for converting externals into submodules or subtrees would be appreciated.
Hello David,
the module for self-hosted GitLab is not yet ready, indeed, but we have it working with GitHub and it’s already possible to test it here:
As for the converting process – I’m afraid we have no such process, what I meant mentioning externals was that the repositories that were configured as externals in your main SVN repository could be translated to Git individually, and then they could be added to the main Git repository as submodules or subtrees, but I’m afraid there is no process or feature that would allow doing that automatically, this is manual process which configuration details strongly depend on the desired setup.
Thank you Ildar, I appreciate your fast responses.
Ildar. The import finished. However I do have another question. I set the [translate] externals = true in the config file for this repository before starting the install process. I’m not seeing a .gitsvnextmodules file anywhere in the repo. Where should I look for this?
Hello David,
the .gitsvnextmodules file is being created in the root of every every Git commit. When you clone the repository, you should see it in the root of your working tree.
@ildar.khusainov I don’t see that content. Its possible the setting was incorrect at point of configuration. Is it possible to re-run the configuration step on a repository that is already being synchronized? How should I approach this?
Hello David,
if the files are absent, then it’s indeed might be caused by incorrect setting. It is possible to add the translate.externals setting at any time, even after the initial import, but in this case it will only be applied to the revisions imported after the setting has been applied; to apply the setting to all the revisions it should be set prior to the initial import, but this cannot be done after the initial import. To apply the setting to a running mirror it should be added to the configuration file and then subgit install should be invoked against the repository:
subgit install REPO_PATH
the REPO_PATH here stands for the path the repository.