Strange issue after updating the subgit mapping file

SVN Mirror
#1

Hi Support,

We have enabled bi-directional sync between SVN and bitbucket for quite some time and it’s working fine.

This morning, one of our staff trying to update the subgit mapping file in order to include a sandbox branch in bitbucket:

This is the original setting:
trunk = trunk:refs/heads/develop
branches = branches/:refs/heads/release/
excludeBranches = branches/_sandboxes
shelves = shelves/:refs/shelves/
excludePath = .gitignore
[translate]
ignores = false

And here is the new change:
trunk = trunk:refs/heads/develop
branches = branches/:refs/heads/release/
branches = branches/_sandboxes/DRS:refs/heads/feature/DRS
excludeBranches = branches/_sandboxes
shelves = shelves/:refs/shelves/
excludePath = .gitignore
[translate]
ignores = false

And click Apply button to update the mapping file.

Initially, it seems working as the feature branch called DRS appear in bitbucket.

However, other staff found that the develop branch history seems to be rollback to early days (as they can see the last commit is 2018 Nov which is abnormal).

Can you advise if this is expected ? And how can we fix this problem ?

regards,
Terry

#2

Hello Terry,
when you update mappings to add previously excluded mapping the process is the following:

  1. SVN Mirror add-on rolls back the history to the moment when this newly added branch didn’t exist yet.
  2. Then it re-translates the history since that moment.

After finishing (2) you should get your history back with newly added branch but temporarily it could look like the history is rolled back. Is that what you observe?

Could you attach svnmirror.log from GIT_REPO/subgit/logs/ directory?

#3

Hi Dmitry,

How long does it takes for step2 ? And what happened when we git clone the develop branch ? It seems that we got the old code when doing git clone/pull from the develop branch.

Also, it’s strange that I can’t find the subgit folder at all !!!

[root@ip-10-30-41-139 logs]# ls -l /var/atlassian/application-data/bitbucket/shared/data/repositories/34/
total 28
-rw-rw-r–. 1 atlbitbucket atlbitbucket 681 Aug 29 2017 config
drwxrwxr-x. 2 atlbitbucket atlbitbucket 17 Jul 3 2017 db
-rw-rw-r–. 1 atlbitbucket atlbitbucket 20 Sep 19 2017 DELETED
-rw-rw-r–. 1 atlbitbucket atlbitbucket 24 Jul 3 2017 HEAD
drwxrwxr-x. 4 atlbitbucket atlbitbucket 84 Jul 3 2017 hooks
drwxrwxr-x. 2 atlbitbucket atlbitbucket 17 Sep 18 2017 info
drwxrwxr-x. 4 atlbitbucket atlbitbucket 34 Jul 5 2017 logs
drwxrwxr-x. 88 atlbitbucket atlbitbucket 4096 Sep 19 2017 objects
-rw-rw-r–. 1 atlbitbucket atlbitbucket 5376 Sep 18 2017 packed-refs
drwxrwxr-x. 8 atlbitbucket atlbitbucket 84 Aug 29 2017 refs
-rw-rw-r–. 1 atlbitbucket atlbitbucket 285 Jul 3 2017 repository-config
drwxrwxr-x. 3 atlbitbucket atlbitbucket 26 Jul 5 2017 stash-refs

Any suggestion on the location of the logs ? (btw, we are trying to rollback the change for one of the repository, there are 4 changed in total, but it seems it takes very very long time).

#4

Add Yann to this ticket

#5

May I ask you what SVN Mirror add-on version and Bitbucket Server version are you using?

It’s very strange repository structure, I have these files and dirs for comparison: “app-info config db HEAD hooks logs objects packed-refs refs repository-config subgit svn”.

There’s one more logs directory, it’s for global logs: BITBUCKET_HOME/log/ , there should be svnmirror.log

Are you sure this is the correct repository? Did you uninstall SVN Mirror add-on from this repository or deleted the repository? I have a guess that ‘DELETED’ has something to do with repository deletion.

#6

Bitbucket server version: 5.7.0
SVN Mirror add-on version: 3.4.1

Sorry. I looked in the wrong directory (as that repository already deleted sometime ago).

Attached is the subgit logs for that repository. (subgit.log.gz)
(note that we are trying to rollback the subgit mapping configuration for this repository).

I attached another one where we didn’t rollback yet (svnmirror_vpbb.zip)
You may want to check this file first.

regards,
Terry

#7

{code}
2019-01-18 05:50:42,872 sync - Restoring refs to reflect SVN state.
2019-01-18 05:50:42,946 sync - Refs were successfully updated
2019-01-18 05:50:42,946 sync - Location ‘/var/atlassian/application-data/bitbucket/shared/data/repositories/93’ fetch completed, fetch took 5005358 ms.
2019-01-18 05:50:42,946 sync - Repository fetch completed, fetch took 5005358 ms.
2019-01-18 05:50:42,950 sync - about to refresh repository
2019-01-18 05:50:42,950 sync - ref delta: 1d296fe8504312435f2cc3c4d9af2f6f72492cbe 82f807e3271507f3ffc83e08e150843e894d78b9 refs/heads/release/1.7.x.0
2019-01-18 05:50:42,950 sync - ref delta: 0000000000000000000000000000000000000000 54c9300545f26ae0914104488e26be934e87b351 refs/heads/feature/DRS
2019-01-18 05:50:42,950 sync - ref delta: 30283ba7371ff767c0eed6fa2a5fe79b1eebdce0 ad0e96e45b55b510cd1f278ea368ba48d404c98b refs/heads/release/1.4.x.0
2019-01-18 05:50:42,950 sync - ref delta: ba8336be6f55ad4bc93ca4580768f203160af794 c2d1157f82ad022b20793f97cb1fecbeba776c4f refs/heads/develop
2019-01-18 05:50:42,950 sync - ref delta: 2933895f85d20bf51964f7a66165f0ce85162b18 0000000000000000000000000000000000000000 refs/heads/release/1.20.x.2
2019-01-18 05:50:42,950 sync - ref delta: 6adc1560a4548d68cf57179b1b044e374a1c8dcb dbd2d7cbd7aadc84ba5272e9e5439e8eff5762b9 refs/heads/release/1.8.x.0
2019-01-18 05:50:42,950 sync - ref delta: c4f35a103aee4f3cbd72b9a170649caca93feb28 8851c9ec7bcf55cf8479d8ea2cd4aecea44c3b31 refs/heads/release/1.5.x.0
2019-01-18 05:50:42,950 sync - ref delta: ee31a3f1aea50f7ea901ad81ffca97486b55a448 dea1c655b7cec6a94793a66500f039117d5cfe0a refs/heads/release/1.6.x.0
{code}

#8

While I’m trying to understand what’s going on, I would suggest a work-around: commit something to SVN directly, to ‘develop’ branch. Then SVN Mirror add-on will update the reference in the Git repository and it will be up to date again.

#9

on which repo ? the repo where config was rollback, the where config doesn’t change or both ?

#10

To the SVN repository corresponding to the repository where the config was rollback. To be more precise: to ‘trunk’ of that repository. If this helps, you can do the same with other branches that are not up to date in Git.

Update: not to branches/develop but to trunk as trunk is configured to be translated to ‘develop’ branch in Git.

#11

OK thanks Dmitry. I try and let you inform of the result

#12

I now see why the issue happens. You wanted to add “branches/_sandboxes/DRS” it appeared first at r35928 so SVN Mirror add-on should have started to re-translate everything since r35928. But you also have “minimalRevision = 180000” and SVN Mirror add-on should have taken this into account but it hadn’t.

Because of this confusion, it also refetched revisions [35928, 179999] and updated Git references in the process. Then it skipped everything but “branches/_sandboxes/DRS” changes on [180000, HEAD] (and this is expected). As result you have branches reset to older values.

Fortunately the work-around I proposed should work (leave a comment if it doesn’t): commit something translatable (e.g. change a file) to every branch that is reset to its older version directly to SVN. Then SVN Mirror add-on will translate this change to Git and update reference to the latest revision. You could also update the references manually, but it’s easier to break something this way.

Meanwhile, I’ll try to reproduce and fix the issue. We’re about to release new SVN Mirror add-on version, it shouldn’t contain this bug.

#13

Thanks for your help, as you indicated, the workaround fix the problem our 3 repos.

#14

r4188 of trunk: a fix for that scenario. I couldn’t reproduce it exactly but found another problem and added a regression test.

#15

Hi @dmitry.pavlenko . We had same issue yesterday on another repo. Is the last version (v3.4.6) containing a fix for this ?
We are currently in v3.4.1.
Thanks
Yann

#16

Hi Yann

sorry for the delay in answer.
3.4.6 does contain some fixes for that matter yet we aren’t completely sure the issue resolved to the very end as we haven’t managed to reproduce it. We are now preparing the next version 3.4.7 (and also 4.0.2 that support Bitbucket 6) and we are hoping to implement the fix in this version. May we have logs from the second affected repository for analysis? This would help us much to find out the cause.

Thank you.