Git object can't be accessed in the Git repository

SubGit
#1

Hi, we’ve started getting this push error after a weekend Gitlab update (13.1) with Subgit 3.3.8:

remote: UNRECOVERABLE ERROR:
remote: Missing unknown fd6797bf748c4133f91315e058be681c2387bc40
remote:
remote: CURRENT STATE:
remote: Git object fd6797bf748c4133f91315e058be681c2387bc40 can’t be accessed in the Git repository /var/opt/gitlab/git-data/repositories/(group)/(project).git.
remote:
remote: POSSIBLE REASONS:
remote: 1) Git server is out of memory, available disk space,
remote: processes count, or open files limits;
remote: 2) When pushing through SSH, the system user on behalf of which.
remote: SubGit daemon is running is not allowed to read files created by SSH user.
remote: TO RECOVER:
remote: 1) Make sure Git server has enough memory, disk resources AND high enough
remote: open handler and processes count limits.
remote: 2) When pushing through SSH, either
remote: A) make sure the SSH user is the same user on behalf of which the SubGit is running
remote: AND
remote: is the same as the system user which is the owner of the Git repository; OR
remote: B) set ‘core.shared’ option to ‘true’ in SubGit configuration file, run
remote: $ subgit install /var/opt/gitlab/git-data/repositories/(group)/(project).git
remote: to apply changes and make sure the owner of the Git repository and SSH user
remote: belong to the same system group.
remote: For details regarding sharing Git repository for a system group see:
remote: https://git-scm.com/docs/git-config#Documentation/git-config.txt-coresharedRepository
remote:
remote: Once you have (1) and (2) fixed, retry push.

Notes:

  1. The same happens for all Subgit-enabled repos.
  2. All Git-only repos accept pushes.
  3. The host is not out of memory/disk/open file descriptors
  4. The repo is owned by “git” user who is also the SSH user (push URL is git@…)
  5. Subgit daemon is running as “git” user (ps -> org.tmatesoft.translator.SubGitDaemon test --svn /var/opt/gitlab/git-data/repositories/(group)/(project).git --limit 1593419575920)
  6. I have enabled core.shared and run “install” but problem persists
  7. The hash is the same as the commit hash (git rev-parse HEAD)
  8. Server git version 2.16.6

Subgit stack trace:

    at org.tmatesoft.translator.daemon.C.a(SourceFile:161)
    at org.tmatesoft.translator.daemon.C.a(SourceFile:144)
    at org.tmatesoft.translator.daemon.C.c(SourceFile:92)
    at org.tmatesoft.translator.daemon.g.a(SourceFile:337)
    at org.tmatesoft.translator.daemon.g.a(SourceFile:45)
    at org.tmatesoft.translator.daemon.g$1.run(SourceFile:308)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

daemon.log:

[2020-06-29 04:53:02.385][daemon][11] Command started, commands running: 1.
[2020-06-29 04:53:02.385][daemon][11] Sent ‘(version (3.3.8 4337 ))’.
[2020-06-29 04:53:02.385][daemon][11] Read '(pre-receive (/var/opt/gitlab/git-data/repositories/(group)/(project).git/. /var/opt/gitlab/git-data/repositories/(group)/(project).git )100:bc2502cd541412f7673e0223271f8ed911d71134 fd6797bf748c4133f
91315e058be681c2387bc40 refs/heads/master

Appreciate any help on this. Thanks.

#2

Hello Ollie,

in GitLab 13.1 they changed some internals that SubGit was relying upon and the upgrade indeed leads to errors like that one at this time. We’re aware of the problem and preparing a new build that will resolve this issue, expect it will be available in a couple of days.
A possible workaround that resolves the issue is to edit SubGit pre-receive hook adding the following line there:

export GIT_QUARANTINE_PATH=$GIT_OBJECT_DIRECTORY

In GitLab environment, SubGit’s hooks reside not in ‘hooks’ directory, but in directory that’s set in hooks.directory setting in the SubGit configuration file. Usually, it’s a directory called ‘custom_hooks’. There should be a file called ‘pre-receive’ – that is the one that should be edited. Insert the line above right before the following:

if [ -x "$C_HOOK" ] ; then

this should resolve the issue and make the pushes possible.

[Enquiry] SubGit version upgrade and SubGit - GitLab version compatibility
#3

Thank you for the rapid reply! I can confirm that has worked for one of the repos, will let you know if any issues with the others.

#4

Hello Ollie,

I’m glad to inform we have released the new version that resolved GitLab and a couple other issues.
The new version is already available on our website:

https://subgit.com/download

To upgrade SubGit first update binaries in SubGit installation directory or upgrade it using *.deb package
and then run ‘subgit install’ against every mirrored repository.