Error: GC overhead limit exceeded

amajumdar2 · February 14, 2021, 7:45am

During Subgit import, after almost 12 hours, the import failed with error:
GC overhead limit exceeded

The .GIT import directory created of size almost 12.5 GB and then the import failed. PFB the commands:

C:\Users\amajumdar2\Documents\SVN2GIT\SubGit\subgit-3.3.10\bin>subgit configure --svn-url http://fcsource01:8080/svn/WICER/GEM GEM.GIT --layout directory
SubGit version 3.3.10 (‘Bobique’) build #4368

Configuring writable Git mirror of remote Subversion repository:
Subversion repository URL : http://fcsource01:8080/svn/WICER/GEM
Git repository location : GEM.GIT

CONFIGURATION SUCCESSFUL

C:\Users\amajumdar2\Documents\SVN2GIT\SubGit\subgit-3.3.10\bin>subgit import GEM.GIT
SubGit version 3.3.10 (‘Bobique’) build #4368
Authentication realm: Subversion Repositories
Username [amajumdar2]: amajumdar2
Password for ‘amajumdar2’:

Translating Subversion revisions to Git commits…

IMPORT FAILED

error: GC overhead limit exceeded
error: Unexpected error has occurred; please report along with the logs (‘C:\Users\amajumdar2\Documents\SVN2GIT\SubGit\subgit-3.3.10\bin\subgit-import-20210212-083810.zip’)

Due to company policy, I am not able to upload the logs. PFB the error logs:

[2021-02-12 08:37:37.374][subgit-import][1] For Trunk/SQA/EntServices/src/11g_services/ChargebackClaimResponse/CreateChgbkClaimRespModelNReqABCSImplV1/classes/scac.log “text” and “eol” are set to “UNSET” and “UNDEF” because
[2021-02-12 08:38:10.499][subgit-import][1] GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.newNode(Unknown Source)
at java.util.HashMap.putVal(Unknown Source)
at java.util.HashMap.put(Unknown Source)
at com.syntevo.svngitkit.core.internal.i.a.C.a(SourceFile:178)
at com.syntevo.svngitkit.core.internal.i.a.A.a(SourceFile:128)
at com.syntevo.svngitkit.core.internal.i.a.A.a(SourceFile:25)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:101)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:49)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:23)
at com.syntevo.svngitkit.core.internal.i.b.a.a(SourceFile:211)
at com.syntevo.svngitkit.core.internal.i.b.s.a(SourceFile:279)
at com.syntevo.svngitkit.core.internal.i.b.s.d(SourceFile:243)
at com.syntevo.svngitkit.core.internal.i.d.a(SourceFile:173)
at com.syntevo.svngitkit.core.internal.i.b.j(SourceFile:164)
at com.syntevo.svngitkit.core.internal.d.o.a(SourceFile:361)
at com.syntevo.svngitkit.core.internal.d.o.addFile(SourceFile:90)
at com.syntevo.svngitkit.core.internal.d.n.addFile(SourceFile:307)
at com.syntevo.svngitkit.core.internal.d.A.addFile(SourceFile:94)
at com.syntevo.svngitkit.core.internal.d.p.addFile(SourceFile:132)
at com.syntevo.svngitkit.core.internal.d.G.addFile(SourceFile:144)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.addFile(SVNCancellableEditor.java:108)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.startElement(DAVEditorHandler.java:415)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.startElement(BasicDAVHandler.java:89)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
[2021-02-12 08:38:10.696][subgit-import][1] Restoring refs to reflect SVN state.
[2021-02-12 08:38:10.734][subgit-import][1] Refs were successfully updated
[2021-02-12 08:38:10.750][subgit-import][1] GC overhead limit exceeded
org.tmatesoft.translator.util.f: GC overhead limit exceeded
at org.tmatesoft.translator.util.f.c(SourceFile:109)
at org.tmatesoft.translator.util.f.b(SourceFile:75)
at org.tmatesoft.translator.k.aj.c(SourceFile:1000)
at org.tmatesoft.translator.k.aj.a(SourceFile:1020)
at org.tmatesoft.translator.k.aj.b(SourceFile:1078)
at org.tmatesoft.translator.k.aj.a(SourceFile:682)
at org.tmatesoft.translator.k.aj.a(SourceFile:651)
at org.tmatesoft.translator.k.d.g.a(SourceFile:244)
at org.tmatesoft.translator.k.d.f.d(SourceFile:47)
at org.tmatesoft.translator.a.I.b(SourceFile:136)
at org.tmatesoft.translator.a.t.c(SourceFile:23)
at org.tmatesoft.translator.b.a(SourceFile:193)
at org.tmatesoft.translator.b.a(SourceFile:144)
at org.tmatesoft.translator.b.a(SourceFile:54)
at org.tmatesoft.translator.process.f.d(SourceFile:136)
at org.tmatesoft.translator.process.f.c(SourceFile:106)
at org.tmatesoft.translator.b.a(SourceFile:79)
at org.tmatesoft.translator.b.a(SourceFile:70)
at org.tmatesoft.translator.SubGit.main(SourceFile:10)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.newNode(Unknown Source)
at java.util.HashMap.putVal(Unknown Source)
at java.util.HashMap.put(Unknown Source)
at com.syntevo.svngitkit.core.internal.i.a.C.a(SourceFile:178)
at com.syntevo.svngitkit.core.internal.i.a.A.a(SourceFile:128)
at com.syntevo.svngitkit.core.internal.i.a.A.a(SourceFile:25)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:101)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:49)
at com.syntevo.svngitkit.core.internal.i.a.v.a(SourceFile:23)
at com.syntevo.svngitkit.core.internal.i.b.a.a(SourceFile:211)
at com.syntevo.svngitkit.core.internal.i.b.s.a(SourceFile:279)
at com.syntevo.svngitkit.core.internal.i.b.s.d(SourceFile:243)
at com.syntevo.svngitkit.core.internal.i.d.a(SourceFile:173)
at com.syntevo.svngitkit.core.internal.i.b.j(SourceFile:164)
at com.syntevo.svngitkit.core.internal.d.o.a(SourceFile:361)
at com.syntevo.svngitkit.core.internal.d.o.addFile(SourceFile:90)
at com.syntevo.svngitkit.core.internal.d.n.addFile(SourceFile:307)
at com.syntevo.svngitkit.core.internal.d.A.addFile(SourceFile:94)
at com.syntevo.svngitkit.core.internal.d.p.addFile(SourceFile:132)
at com.syntevo.svngitkit.core.internal.d.G.addFile(SourceFile:144)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.addFile(SVNCancellableEditor.java:108)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.startElement(DAVEditorHandler.java:415)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.startElement(BasicDAVHandler.java:89)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)

Regards,
Abhishek

ildar.khusainov · February 15, 2021, 4:10am

Hello Abhishek,

this error means that SubGit ran out of memory performing the initial import. Normally SubGit consumes little memory and doesn’t much processing. But there’s one feature - EOLs translation - that really performs line endings translation for each file, this can affect performance. Also large .gitattributes file may be produced, this also leads to large RAM consumption. To improve performance you can turn the feature off.

[translate]
    eols = false
    otherProperties = false

You can also try to change the following option:

[core]
    streamFileThreshold = 20000000

It changes the way SubGit works with Git blobs: loads fully to memory or streams byte-by-byte from disk. Normally full loading is faster but may result into OutOfMemoryError. Also when streaming, SubGit tries to use ‘git’ command line to do that (faster) — it tries to find ‘git’ in PATH or using

[core]
    gitPath = /path/to/git/executable

option. But if it can’t find Git, it uses internal means of streaming and they are much much slower.