Hi,
We use Subgit in our Gitlab installation and it has worked well for months.
Lately it has been taking longer and longer to create a merge request in git lab (I believe this creates a branch in git and therefore in svn).
It is taking 40 seconds at least each time and sometimes causes gitlab to time out. We diagnosed this using a top command on the server command line a Java process which we believe is the mirror
Do you have any advice on how we might increase the performance of the mirror?
Hi Steve,
a branch creating may indeed take time, especially if the branch consists many commits, so it can be the reason for the delays, yet I’m not completely sure how exactly it works in your setup, my understanding was a feature branch is created at some time, translated to SVN immediately, and a merge request just merges that branch into, say, master thus not creating any branches, isn’t that a process behind the merge requests in your setup?
Anyway, I don’t recall any issues specific to merge requests. There are some specifics that may affect performance (like EOLs translating) but they are common and not only specific to the merges. So I think this would require a deeper look, could it be possible to collect all the SubGit logs from an affected repository right after another long merge request?
When I hit merge request it creates a branch of the parent branch and then commits it. It takes approximately 40 seconds though which seems a very long time.
Which logs would you like me to provide? I can do it now?
Here are two that are usually useful, let me know if you need others.
pre-receive-hook.0.log (182.3 KB)
daemon.0.log (2.9 MB)
Hi Steve,
could you please also share the time when the issue occurred last time and branches names, if possible.
Of course:
The time was approximately 9:30am this morning.
The branch name is 9877-qa_test-17
Hi Steve,
thank you for the logs and clarification.
From the logs it looks like you have plenty of the references (branches and tags) in the repository, is that the case for this exact repository? If so, then that is the reason of the issue since SubGit performs references check every fetch and as more references in the repository the longer this process is. The best solution for it would be reducing the number of references – if, for example, you have many branches that are not needed already. removing those branches will reduce the fetch (and thus pull requests merge) time.
A possible workaround if branches removal is not possible is to try to pack all those references with the git pack-refs command (possibly, better with --all option), this should reduce the time needed for SubGit’s references check.
Hi,
We have deleted several hundred branches out of our 3318 and pruned the server and packed the refs.
It has not made a noticeable difference.
We have 14450 tags and it looks like we might be able to remove 10000 or so of them.
Will this provide a speed increase for us?
Hi Steve,
3K branches and 15K tags is indeed very much, just hundreds of references may not be that noticeable against those thousands. Removing 10K out of 15 should provide a noticeable increase, yes.
@Steve_Taylor You might want to rethink that approach: you’re denying your team the ability to get the most out of feature branches while giving them all the pain of git, svn and git&svn.
One of the most valuable advantages of using a git/requests-based workflow is when you use a rebase or rebase & squash before merging. Any CI you have on the branch will get rerun and you can merge it with much greater confidence.
I’d strongly encourage you to review having feature/task branches be short and transient, and prefer your users to squash before rebasing, rebase before merging, and delete branch on merge.
Both your SVN and git users will thank you.
I totally agree but I am bound by large amounts of technical debt.
Will not squashing branches cause the mirror to perform more poorly?