Hi Anton,
yes, for SVN and Git log I meant output of the log command, yet I’d also ask to add -v in the SVN log.
Hi Ildar.
The svn with a -v key contains sensitive information. Could you please tell me what exactly you need from the log? Probably, we could keep the information that you need and hide the rest that we don’t ready to show.
Hi Anton,
we need to know what exactly has been changed in every particular revision – that is, if some files were modified, or added, or a new branch created out of another branches, etc. We don’t need particular commit messages or even filenames.
Hi Ildar.
Sorry, let me clarify this. You need to know what exactly changed but you don’t need a diff?
We don’t create new branches, but I am not sure how to avoid showing you the diff, but show you the rest.
It would be helpful if you give me examples of what information are you expecting.
For example, for git, I could provide you with this:
commit e08413c66bb79e3faaee05140a78f939114f62f9
Author: Developer Name <dName@meraki.com>
xxxxxx commit message xxxxxxxx
Change-Id: Iaa09c9ea2e0b718e2a98409ae59d0eb775598fc1
Reviewed-on: https://gerrit.server.com/c/manage/+/191568
Reviewed-by: Another Developer <aDeveloper@meraki.com>
Style-Verified: Jenkins Build System <jenkins@meraki.com>
Tests-Verified: Jenkins Build System <jenkins@meraki.com>
for SVN:
Hi Anton,
sorry for being not clear enough.
We definitely do not need the actual changes, only need to understand what kind of change has been made in a new revision or commit – just some files modification, or some paths copying, etc., like this for SVN:
r5794 | root | 2019-05-31 20:42:20 +0500 (Пт, 31 май 2019) | 1 line
Changed paths:
A /pr4
A /pr4/branches
A /pr4/tags
A /pr4/trunkHello!
I’m working with Anton, and we were able to identify a few issues after performing several heap dumps. The main problem appears to be threads not terminating correctly. In the heap dump we took where subgit had exhausted its allocated 32GB of heap space, we saw 7378 threads in the waiting state . VisualVM showed them as "Piping consumer" daemon prio=5 tid=21411 TIMED_WAITING.
"Piping consumer" daemon prio=5 tid=23977 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at org.tmatesoft.svn.core.internal.io.svn.StreamLogger.lambda$new$0(StreamLogger.java:56)
local variable: org.tmatesoft.svn.core.internal.io.svn.StreamLogger#7342
local variable: java.util.logging.Logger#8
local variable: java.util.logging.Level#5
local variable: com.trilead.ssh2.channel.ChannelInputStream#14705
local variable: java.lang.String#744171
local variable: byte[]#124998
at org.tmatesoft.svn.core.internal.io.svn.StreamLogger$$Lambda$53.run(<unresolved string 0x0>)
at java.lang.Thread.run(Thread.java:750)
Opening the heap dump in Eclipse’s Memory Analyzer plugin totalled the occupied heap space from those threads at 28.8GB.
The stack trace of the stuck threads was identical to the problem identified in this SVNKit support topic: Replacing trilead-ssh2 with Apache sshd - #9 by joel-wenzel and I suspect that we’re seeing the same issue. If I understand the conversation in that topic, there was a fix pushed out with SVNKit 1.10.6. Our current version of Subgit (3.3.14) seems to be the latest, but it looks like it uses SVNKit 1.10.5:
[2022-06-02 10:49:42.999][subgit-install][1] SubGit version 3.3.14 ('Bobique') build #4433
[2022-06-02 10:49:43.000][subgit-install][1] System properties:
java.runtime.name : OpenJDK Runtime Environment
sun.boot.library.path : /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64
java.vm.version : 25.332-b09
java.vm.vendor : Oracle Corporation
java.vendor.url : http://java.oracle.com/
path.separator : :
java.vm.name : OpenJDK 64-Bit Server VM
file.encoding.pkg : sun.io
user.country : US
sun.java.launcher : SUN_STANDARD
sun.os.patch.level : unknown
jna.nosys : true
java.vm.specification.name : Java Virtual Machine Specification
user.dir : <redacted>
java.runtime.version : 1.8.0_332-8u332-ga-1~deb8u1-b09
java.awt.graphicsenv : sun.awt.X11GraphicsEnvironment
java.endorsed.dirs : /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed
os.arch : amd64
java.io.tmpdir : /tmp
line.separator :
java.vm.specification.vendor : Oracle Corporation
os.name : Linux
sun.jnu.encoding : UTF-8
svnkit.http.methods : Digest,Basic,NTLM,Negotiate
java.library.path : /usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-
gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
java.specification.name : Java Platform API Specification
java.class.version : 52.0
sun.management.compiler : HotSpot 64-Bit Tiered Compilers
svnkit.ssh.client : apache
sun.io.useCanonCaches : false
os.version : 4.9.0-0.bpo.6-amd64
user.home : <redacted>
user.timezone : America/Los_Angeles
java.awt.printerjob : sun.print.PSPrinterJob
file.encoding : UTF-8
java.specification.version : 1.8
java.class.path : /usr/share/java/subgit/jansi-1.6.jar:/usr/share/java/subgit/slf4j-nop-1.7.12.
jar:/usr/share/java/subgit/unirest-java-1.3.0.jar:/usr/share/java/subgit/javax.ws.rs-api-2.1.jar:/usr/share/j
ava/subgit/jetty-server-9.4.11.v20180605.jar:/usr/share/java/subgit/jetty-servlet-9.4.11.v20180605.jar:/usr/s
hare/java/subgit/jersey-server-2.27.jar:/usr/share/java/subgit/jersey-hk2-2.27.jar:/usr/share/java/subgit/jer
sey-container-jetty-servlet-2.27.jar:/usr/share/java/subgit/jersey-media-json-jackson-2.27.jar:/usr/share/jav
a/subgit/svnkit-1.10.5-snapshot20220408143050.jar:/usr/share/java/subgit/org.eclipse.jgit-5.0.11-sgk.jar:/usr
/share/java/subgit/annotations-7.0.3.jar:/usr/share/java/subgit/httpclient-4.3.jar:/usr/share/java/subgit/htt
pasyncclient-4.0-beta4.jar:/usr/share/java/subgit/httpmime-4.3.jar:/usr/share/java/subgit/json-20090211.jar:/
usr/share/java/subgit/javax.servlet-api-3.1.0.jar:/usr/share/java/subgit/jetty-http-9.4.11.v20180605.jar:/usr
/share/java/subgit/jetty-io-9.4.11.v20180605.jar:/usr/share/java/subgit/jetty-security-9.4.11.v20180605.jar:/
usr/share/java/subgit/jersey-common-2.27.jar:/usr/share/java/subgit/jersey-client-2.27.jar:/usr/share/java/su
bgit/jersey-media-jaxb-2.27.jar:/usr/share/java/subgit/javax.annotation-api-1.2.jar:/usr/share/java/subgit/ja
vax.inject-2.5.0-b42.jar:/usr/share/java/subgit/validation-api-1.1.0.Final.jar:/usr/share/java/subgit/hk2-loc
ator-2.5.0-b42.jar:/usr/share/java/subgit/jersey-container-servlet-2.27.jar:/usr/share/java/subgit/jersey-con
tainer-jetty-http-2.27.jar:/usr/share/java/subgit/jetty-webapp-9.4.7.v20170914.jar:/usr/share/java/subgit/jer
sey-entity-filtering-2.27.jar:/usr/share/java/subgit/jackson-annotations-2.8.10.jar:/usr/share/java/subgit/ja
ckson-databind-2.8.10.jar:/usr/share/java/subgit/jackson-module-jaxb-annotations-2.8.10.jar:/usr/share/java/s
ubgit/sshd-core-2.8.0.jar:/usr/share/java/subgit/sshd-common-2.8.0.jar:/usr/share/java/subgit/eddsa-0.3.0.jar
:/usr/share/java/subgit/sequence-library-1.0.4.jar:/usr/share/java/subgit/sqljet-1.1.14.jar:/usr/share/java/s
ubgit/jna-5.6.0.jar:/usr/share/java/subgit/jna-platform-5.6.0.jar:/usr/share/java/subgit/trilead-ssh2-1.0.0-b
uild222.jar:/usr/share/java/subgit/jsch.agentproxy.connector-factory-0.0.7.jar:/usr/share/java/subgit/jsch.ag
entproxy.svnkit-trilead-ssh2-0.0.7.jar:/usr/share/java/subgit/lz4-java-1.4.1.jar:/usr/share/java/subgit/jsch-
0.1.54.jar:/usr/share/java/subgit/JavaEWAH-1.1.6.jar:/usr/share/java/subgit/bcpg-jdk15on-1.62.jar:/usr/share/
java/subgit/bcprov-jdk15on-1.62.jar:/usr/share/java/subgit/bcpkix-jdk15on-1.62.jar:/usr/share/java/subgit/htt
pcore-4.3.jar:/usr/share/java/subgit/commons-logging-1.1.3.jar:/usr/share/java/subgit/commons-codec-1.6.jar:/
usr/share/java/subgit/httpcore-nio-4.3-beta2.jar:/usr/share/java/subgit/jetty-util-9.4.11.v20180605.jar:/usr/
share/java/subgit/osgi-resource-locator-1.0.1.jar:/usr/share/java/subgit/aopalliance-repackaged-2.5.0-b42.jar
:/usr/share/java/subgit/hk2-api-2.5.0-b42.jar:/usr/share/java/subgit/hk2-utils-2.5.0-b42.jar:/usr/share/java/
subgit/javassist-3.22.0-CR2.jar:/usr/share/java/subgit/jersey-container-servlet-core-2.27.jar:/usr/share/java
/subgit/jetty-continuation-9.4.7.v20170914.jar:/usr/share/java/subgit/jetty-xml-9.4.7.v20170914.jar:/usr/shar
e/java/subgit/jackson-core-2.8.10.jar:/usr/share/java/subgit/jcl-over-slf4j-1.7.32.jar:/usr/share/java/subgit
/antlr-runtime-3.4.jar:/usr/share/java/subgit/jsch.agentproxy.core-0.0.7.jar:/usr/share/java/subgit/jsch.agen
tproxy.usocket-jna-0.0.7.jar:/usr/share/java/subgit/jsch.agentproxy.usocket-nc-0.0.7.jar:/usr/share/java/subg
it/jsch.agentproxy.sshagent-0.0.7.jar:/usr/share/java/subgit/jsch.agentproxy.pageant-0.0.7.jar:/usr/share/jav
a/subgit/javax.inject-1.jar:/usr/share/java/subgit/slf4j-api-1.7.32.jar:/usr/share/java/subgit/translator-3.3
.14.jar:/usr/share/java/subgit/svngitkit-2.3.4-SNAPSHOT_877c759.jar
In case it’s hard to read, what I’m referring to is the /usr/share/jav a/subgit/svnkit-1.10.5-snapshot20220408143050.jar in the java.class.path
Would it be possible to get a version of Subgit with an SVNKit version bump to 1.10.6? Alternatively, is there a manual workaround that we can complete ourselves?
Thanks for the information. Indeed this problem seems to be the same as Joel Wenzel’s. As I can see from svnkit-1.10.5-snapshot20220408143050.jar filename, SVNKit version corresponds to r10827 (committed on Fri Apr 8 15:40:47 2022). But the fix to the Joel Wenzel’s problem was done at r10828. So SVNKit >=1.10.6 could be the solution. We’re planning to release a new SubGit version with the latest SVNKit (it will be 1.10.7) at the beginning of the next week.
If you don’t want, you can use the following trick:
- Backup
GIT_REPO/subgit/lib
directory.
- Copy all JAR files from SubGit distribution’s
lib/subdirectory to
GIT_REPO/subgit/lib
directory.
- Replace the SVNKit jar in
GIT_REPO/subgit/lib
with SVNKit 1.10.6
- Edit
GIT_REPO/subgit/lib/.files
file to enumerate there all JAR files and libjnidispatch.so file using <filename>=<SHA-1> format. Except the “fat jar” file (subgit-XXX_fat.jar) — delete or comment out that line. You can use the following one-liner to do that:
for file in *; do echo -n $file=; sha1sum $file | cut -f 1 -d ' '; done
- Delete the “fat jar” from
GIT_REPO/subgit/lib/
directory.
It’s a general trick to replace any JAR. The “fat jar” just consists of all those JARs unpacked and piled together to a single JAR file, so you can just replace it with many JAR files.
And .files file is used for consistency check: if some file’s SHA-1 doesn’t match its record in .files file, “subgit install” tries to “repair” the installation by replacing all the JARs with a “fat jar” it creates from its distribution. So it’s important to have correct SHA-1 hashes there. This list is also used to load upon start.
Anyway it would be easier just to wait for a new release, I hope it will fix the problem.