Subgit failing to sync a single folder from svn repository

SubGit
#1

We have discovered that subgit is not synchronising all files that were in our svn repository into the new git repository. I’ve catalogued all of the files and the only ones which are missing (excluding some empty directories that were added to svn, but that git can’t produce) are inside a single folder.

The folder is <repository-root>/hurricaneRootfs/opt/DXi/systemd/, and contains some text scripts. The two folders up in the hierachy are otherwise empty, ie: the opt/ folder only contains the DXi folder. The repository currently does not have a .gitignore file, and this isn’t one of the excluded directories in the subgit config. Other files/folders in the hurricaneRootfs/ do exist in the git repository and are being synchronised correctly.

Things I’ve tried to force a sync:

  • adding a newline to the end of each of the missing files and commiting to svn
  • adding a new dummy text file into the same folder and committing to svn

I can see from the subgit logs that subgit is detecting the commits, but no git commits are being generated. Anything in this location just gets swallowed and git won’t reproduce the opt/DXi/systemd/ or any of its contents. What is going on here? What can I do to resolve this?

Thanks, Jono

1 Like
#2

Hello Jono,

thanks for reaching out to us on that matter!

May I ask you to collect all the SubGit logs from the affected repository and attach them to this thread for us to analyse? It would also be great to have a couple of filenames that exist in SVN but are not synchronized to Git.

Thank you in advance!

#3

Thankyou for the fast response. I will grab the logs and get back to you.
Is there any identifying data within the logs that I will need to anonymise before sharing them publicly?

#4

by default, SubGit logs the committers/authors names and emails along with the commit hashes, revision numbers, and commit messages. Also, in case of a fault SubGit usually puts complete configuration file in the log, and the config may contain repository url and credentials.

#5

Okay. I don’t think I’m going to be permitted to share that information publicly. Please contact me at jono.targett@quantum.com, and I will reply with the logs as an attachment. We can then continue the discussion here.

#6

Hello Jono,
I think, the problem is that you have

excludePath = DXi

in the config file. If you look at svn.excludePath documentation (the article it refers to), you’ll see that “DXi” pattern is excluded recursively in each branch or tag. As you say, “opt” directory only contains “DXi”, thus after excluding “DXi”, the “opt” directory becomes empty and thus is not preserved in Git either.

I should also note that excluding a path from translation means that any changes to that path on SVN side will not be translated to Git and any changes to it on Git side will not translated to SVN. I.e. such paths effectively become unlinked. That’s why you can change “DXi” from Git side without observing any effect on the SVN side.

As you can see from the documentation, there’re also ways to exclude paths non-recursively. E.g. if you wanted to exclude <branch-root>/hurricaneRootfs/opt/DXi non-recursively, you could use

excludePath = /hurricaneRootfs/opt/DXi

The leading slash makes a pattern non-recursive. In this example inside each branch or tag /hurricaneRootfs/opt/DXi path would be excluded from translation.

Also I would like to pay your attention to that you write about <repository-root>/hurricaneRootfs/opt/DXi/systemd/. If by <repository-root> that you mean branch or tag root, that’s ok. But if you mean the project root defined by svn.url option, I should say that only paths specified in trunk/branches/tags/shelves options are translated to Git, everything beyond is ignored.

Finally, I would like to note that one cannot “unexclude” a path without re-translating the whole repository. This is because the older Git commits should then contain paths that were previously excluded. The only changes to excludePath and includePath options that are allowed are the changes that narrowing down the set of translated paths, but not those making it wider.

To re-translate the whole history you can fix your config to unexclude paths you want to be translated and run

subgit install --rebuild path/to/git/repository

or you can just re-create another mirror from scratch with correct settings. In ether cases SHA-1 hashes will be different as the commits will then contain files you need.

I hope this information helps.

#7

Thankyou Dmitry, I don’t know how we missed that! I’ll do the rebuild and then report back.