Hi Ildar,
I have hooks on svn that I want to migrate and convert to Bitbucket hooks.
I have migrated svn repositories to Bitbucket already.
Can you let us know the process?
Thanks,
Sitaram
Hi Sitaram,
I’m afraid SVN Mirror add-on does not migrate hooks, it only takes care of data migration from SVN to Git, hooks should be re-created manually.
Hi Ildar,
Does the migrated repo on Bitbucket not show empty folders in the UI?
The svn has few empty folders as shown below.
However the migrated Bitbucket repository does not show the “lib” and “output” folders as shown below.
Is this expected behaviour? Is there anything that can be done to show the empty folders as well?
Thanks,
Sitaram
Hi Sitaram,
yes, that’s expected behaviour that comes from Git itself, not from SVN Mirror add-on or Bitbucket server. The matter is that Git does not support empty folders:
https://git.wiki.kernel.org/index.php/GitFaq#Can_I_add_empty_directories.3F
A workaround is to add some stub file to the directory that should be present in Git (say, zero-size .empty file) – in this case, the directory will appear in Git.
Hi Ildar,
I get the following error when I try to commit code to a branch on bitbucket repo.
git commit -m "Testing the pre-commit"
Error: Unable to initialize main class com.nvls.ncf.utils.svn.hook.VidValidatorHook
Caused by: java.lang.NoClassDefFoundError: org/tmatesoft/svn/core/wc/ISVNOptions
Commit Failed- You cannot check in as the committed vid files are not same in trunk.
Error: Unable to initialize main class com.nvls.ncf.utils.svn.hook.AlarmValidatorHook
Caused by: java.lang.NoClassDefFoundError: org/tmatesoft/svn/core/wc/ISVNOptions
Commit Failed- You cannot check in as the committed alarm files are not same in trunk.
Can you please check?
Thanks,
Sitaram
Hello Sitaram,
do I understand correctly this is the client-side pre-commit hook that works in a working copy on a developer’s machine? If so, could you please advise what exactly does this pre-commit hook do? Judging from the message, it looks as if the hook uses SVNKit for some tasks and SVNKit is set incorrectly, could it be the case?
If that is no a client-side hook, could you please describe in a little more detail what is that hook and how has it been installed to the repository.
Hi Ildar,
To elaborate on this, it is actually a server side hook. In the current SVN, the server where the repository hosted, they have a pre-commit hook file in the server and that file is a shell script which is calling couple of jars. When the condition is met, it is allowing commit otherwise it rejects.
Is there a way the same kind of pre-commit hooks are possible in bitbucket ?
Thanks
Venkat
Hi,
I’m sharing the jstack logs from our prod bitbucket server. Can you please analyse why it is taking long time for sync.
Thanks
Venkat
Jstack ouput (1).txt (57.5 KB)
Hi Venkat,
Bitbucket server does provide hooks functionality and, generally speaking, it’s possible to set a hook like this in Bitbucket. However, I would’t recommend doing it when SVN Mirror is installed in a repository. The matter is that SVN Mirror add-on also uses hooks to perform some of the synchronization operations, but despite Bitbucket does have hooks, it does not have an API that would provide a way to guarantee the hooks invoking order, so if there is a custom hook in additions to SVN Mirror’s then it’s no possible to predict which one will run first at any given time.
Returning to the initial question about those java.lang.NoClassDefFoundError exceptions – those look to be exceptions coming from the SVN side then: SVN Mirror acts as a regular SVN client sending changes from Git to SVN and thus it also triggers any hooks on SVN side. Apparently, in this case the SVN hook was unable to run and so those scripts and/or jars on SVN side are to be investigated.
As for the long sync – could you please additionally collect SVN Mirror and Bitbucket logs from the affected repository? It would also be great to understand to have an example of a long synchronization.
build.svnmirror.log (414.9 KB)
Hi ildar,
Please find the attached logs.
Thanks
Venkat
svnmirror_build.log (412.6 KB)
svnmirror_configcore.log (5.3 MB)
svnmirror_configpecvd.log (9.8 MB)
svnmirror_src.log (1.6 MB)
Hello Venkat,
thank you for the logs!
One more question about the jstack output: dev team’s feedback is that it looks as if it was collected not from a Bitbucket process but from an unrelated Java process. Could you try to re-collect it making sure running Bitbucket server pid is set as stacks argument and also collect jstack from both Bitbucket nodes, better at the time a synchronization is in progress.
Thank you in advance!
Node1-Jstacklogs.txt (256.0 KB)
Node2-jstacklogs.txt (242.0 KB)
Hi ildar,
Please find the attached jstack logs from both the nodes.
Thanks
Venkat
Hi ildar,
Do you have an update on sync issue ?
Thanks
Venkat
Hello,
we have analyzed logs you have provided. There is indeed an unexpected slowdown for the most simple operations that are expected to be completed in mostly no time:
...
2023-02-15 06:55:03,149 sync - Restoring refs to reflect SVN state.
2023-02-15 07:09:27,717 sync - Refs were successfully updated
...
This part takes mostly 14 minutes while there are no refs to update in that sync operation.
This code (references restoration) only works with local files or, in Data Center case, with Git repository files that are accessed via NFS (network share), in particular files at /bitbucket_home/shared/data/subgit/repositories/1223 path.
So far we have no idea, why file access is that slow. jstack snapshot also displays app thread to stuck (or at least caught) at file access operation.
First thing to try would be to switch SVN Mirror App “Git Backend” option to “Command line” to check if that would resolve the problem. To change this option, go to:
Bitbucket Administration → Manage apps → SVN Mirror for Bitbucket Server → Configure → Add-On Settings
Change Git Backend option to “Command line” and then click Apply to save changes. Then check if performance problem persist.
Also I would recommend to check if there is anything special regarding NFS volume configuration - volume where Git repositories are stored.
Hi Alexander,
We have changed the SVN Mirror app settings from “Git Backend” option to “Command line”, but seems the sync performance is too poor now.
Attaching the logs for reference. Please let me know if we are seeing the correct logs ? and this is taking ~27mins ?
Thanks
Venkat
svnmirror-13-March-2023.log (572.0 KB)
Hi ildar/Alexander,
Any update please ?
Thanks
Hello Santosh,
I’m very sorry for delay with the reply. From the last log you’ve sent indeed it became even slower with the command line option. This part takes the longest time:
2023-03-12 20:27:42,261 sync - Restoring refs to reflect SVN state.
2023-03-12 22:12:30,200 sync - Refs were successfully updated
It doesn’t even actually update refs as it should be a no-op operation in that case, but nevertheless it takes mostly 40 minutes to do so.
What SVN Mirror does is iterates over all repository refs and checks whether it need to be updated.
First it gets list of all refs by running “git for-each-ref” command in “/bitbucket_home/shared/data/subgit/repositories/1223/” directory.
Could you please log in (with ssh) to any of the nodes and run the following:
$ cd /bitbucket_home/shared/data/subgit/repositories/1223/
$ sudo -u BITBUCKET_USER git for-each-ref
And check if that command takes a long time to run. How many refs does it report?
Hi @alexander.kitaev ,
I must have missed the previous comment due to delay in response. Please find the logs that you have asked for from one of our bitbucket node. Please provide your advise on this.
-bash-4.2$ cd /bitbucket_home/shared
-bash-4.2$ cd data/subgit/repositories/1223
-bash-4.2$ git for-each-ref
fatal: not a git repository (or any parent up to mount point /bitbucket_home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Thanks
Venkat