Push failed: missing necessary object *****. Check available Git server memory, running processes count and available disk space and retry push operation

Hi,
After updating Gitlab from version 12.0.2-ce.0 to 13.3.5-ce.0 we’re not able to push anything into synced git repositories.

Git push:
georgysh@mbp:~/Documents/git/zf3$ git push

Enumerating objects: 9, done.

Counting objects: 100% (9/9), done.

Delta compression using up to 8 threads

Compressing objects: 100% (5/5), done.

Writing objects: 100% (5/5), 490 bytes | 490.00 KiB/s, done.

Total 5 (delta 3), reused 0 (delta 0)

remote: + cygwin=false

remote: + os400=false

remote: + darwin=false

remote: + case “uname” in

remote: ++ uname

remote: + PRG=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive

remote: + ‘[’ -h /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive ‘]’

remote: ++ dirname /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive

remote: + abs_path /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks SCRIPT_DIR

remote: + pushd /var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + local result=SCRIPT_DIR

remote: + local file_basename=custom_hooks

remote: + local file_parent=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/

remote: + ‘[’ x/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/ ‘!=’ x ‘]’

remote: + cd /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/

remote: ++ pwd -P

remote: + local abs_file_parent=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local abs_file_parent_with_basename=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + abs_file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + eval ‘SCRIPT_DIR=’’’/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks’’’’

remote: ++ SCRIPT_DIR=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + popd

remote: ++ dirname /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + abs_path /var/opt/gitlab/git-data/repositories/clientside/zframework3.git GIT_REPOS

remote: + pushd /var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local result=GIT_REPOS

remote: + local file_basename=zframework3.git

remote: + local file_parent=/var/opt/gitlab/git-data/repositories/clientside/

remote: + ‘[’ x/var/opt/gitlab/git-data/repositories/clientside/ ‘!=’ x ‘]’

remote: + cd /var/opt/gitlab/git-data/repositories/clientside/

remote: ++ pwd -P

remote: + local abs_file_parent=/var/opt/gitlab/git-data/repositories/clientside

remote: + local abs_file_parent_with_basename=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + abs_file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + eval ‘GIT_REPOS=’’’/var/opt/gitlab/git-data/repositories/clientside/zframework3.git’’’’

remote: ++ GIT_REPOS=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + popd

remote: + GIT_REPOS=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + export JAVA_HOME=/usr/java/jdk1.8.0_202/jre

remote: + JAVA_HOME=/usr/java/jdk1.8.0_202/jre

remote: + JAVA_EXE=/usr/java/jdk1.8.0_202/jre/bin/java

remote: + JAVA_OPTIONS=’-noverify -Xint -Djava.awt.headless=true -Djna.nosys=true’

remote: + MAIN_CLASS=org.tmatesoft.translator.SubGitHook

remote: + TS_CLASSPATH=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/…/subgit/lib/subgit-3.3.6_4276_fat.jar

remote: + false

remote: ++ dirname /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive

remote: + abs_path /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks HOOKS_DIRECTORY

remote: + pushd /var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + local result=HOOKS_DIRECTORY

remote: + local file_basename=custom_hooks

remote: + local file_parent=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/

remote: + ‘[’ x/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/ ‘!=’ x ‘]’

remote: + cd /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/

remote: ++ pwd -P

remote: + local abs_file_parent=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git

remote: + local abs_file_parent_with_basename=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + abs_file=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + eval ‘HOOKS_DIRECTORY=’’’/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks’’’’

remote: ++ HOOKS_DIRECTORY=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks

remote: + popd

remote: ++ cat

remote: + HOOK_INPUT=‘8a59daecea1d0c6f79c7402ae356b11a18eebd80 957decc96591726660abf54579b46817b70b23aa refs/heads/master’

remote: + USER_HOOKS_PREFIX=user

remote: ++ basename /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive

remote: + USER_HOOK=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/user-pre-receive

remote: + ‘[’ -x /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/user-pre-receive ‘]’

remote: + ARGUMENTS=

remote: + umask 0022

remote: + C_HOOKS_PREFIX=fast

remote: + C_HOOK_EXTENSION=

remote: ++ basename /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/pre-receive

remote: + C_HOOK=/var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/fast-pre-receive

remote: + ‘[’ -x /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/fast-pre-receive ‘]’

remote: + echo ‘8a59daecea1d0c6f79c7402ae356b11a18eebd80 957decc96591726660abf54579b46817b70b23aa refs/heads/master’

remote: + /var/opt/gitlab/git-data/repositories/clientside/zframework3.git/custom_hooks/fast-pre-receive

remote:

remote: SubGit ERROR REPORT (SubGit version 3.3.6 (‘Bobique’) build #4276):

remote:

remote: You’ve received this message because SubGit (http://subgit.com/) is installed in your repository

remote: and an error that needs to be dealt with has occurred in SubGit translation engine.

remote:

remote: TEMPORARY ERROR:

remote: Push failed: missing necessary object 957decc96591726660abf54579b46817b70b23aa. Check available Git server memory,

remote: running processes count and available disk space and retry push operation.

Hi Roman,

this is known issue cause by GitLab 13.1 – in GitLab 13.1 they changed some internals that SubGit was relying upon and the upgrade indeed leads to errors like that one. This is resolved in the SubGit version 3.3.10:

https://subgit.com/download

If upgrade is not possible or desired, a possible workaround that resolves the issue is to edit SubGit pre-receive hook adding the following line there:

export GIT_QUARANTINE_PATH=$GIT_OBJECT_DIRECTORY

In GitLab environment, SubGit’s hooks reside not in ‘hooks’ directory, but in directory that’s set in hooks.directory setting in the SubGit configuration file. Usually, it’s a directory called ‘custom_hooks’. There should be a file called ‘pre-receive’ – that is the one that should be edited. Insert the line above right before the following:

if [ -x "$C_HOOK" ] ; then

this should resolve the issue and make the pushes possible.