I am using svnkit java library to access VisualSVN. The initial authentication works but get following exception when checkout the code,
org.tmatesoft.svn.core.SVNException: svn: E175002: chunked stream ended unexpectedly
svn: E175002: REPORT request failed on ‘/svn/FOS_US-ETD/!svn/vcc/default’
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:112) ~[svnkit-1.10.7.jar:na]
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:96) ~[svnkit-1.10.7.jar:na]
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:765) ~[svnkit-1.10.7.jar:na]
have tried both 1.7.14, 1.10.8, same error. Can someone please share a light?
Hello,
when the server sends “Transfer-Encoding: chunked” header, ChunkedInputStream class is used to read the data from the socket.
To read the chunk size, ChunkedInputStream#getChunkSizeFromInputStream method is used. If the connection gets unexpectedly closed at this moment, you see this error.
So this error is a synonym of “the connection is unexpectedly closed”. Why can this happen?
- in case of unexpected network error;
- in case when a bug happens on the server side while sending the response to a client, so that the connection unexpectedly dies;
- when you run into some limits (e.g. bytes count limit or timeout) on the server side, so that the server cuts the connection;
- there could be other reasons why the server cuts the connection, which I don’t know about.
If the problem is not reproducible and happens from time to time, probably, it’s network error. If the problem is more or less reproducible, but not 100%, it could be a timeout issue. If it’s 100% reproducible, it’s worth sniffing the network traffic, e.g. with WireShark. If the problem happens over and over again while sending the same file, probably, the problem is near this file itself. Then it’s worth comparing network traffic for native SVN and SVNKit.
I hope you get the idea. And of course there’s always a chance that there’s some bug in ChunkedInputStream class, but it seems to be simple and every time a problem of this kind had been reported, it was related to a network connection issue.
Thanks Dmitry, this is helpful. It happens every time when I try to checkout, regardless of which project/files are being checked out. The chunk size as you mentioned, is it a value setup on the VisualSVN server side?
Also just to add more info, I can check out from the VisualSVN using Eclipse SVN or other SVN client without problem. It seems only happens when I check through the Java/svnkit process.
Then there’s a chance that it’s SVNKit that reads the chunked stream incorrectly. In this case it would be perfect if we could fix this problem as we got similar reports before.
I don’t know how to setup the chunk size using Apache. I would suspect, the chunk size is chosen by the server dynamically when passing the data stream through a compression filter like gzip.
Could I ask you to assist me in fixing this problem once and forever? If yes, I would ask you to record the traffic either with Wireshark (if it’s not compressed) or with SVNKit itself. For the latter use java.util.logging.config.file JVM property and specify a file logging property file like this one. You can replace “svnkit.%u.log” with some absolute path pattern, otherwise a “current” directory is used (new File("").getAbsolutePath()). This will dump the network traffic. I expect that at some point it will end with some HTTP response with some chunk size SVNKit processes incorrectly. Or maybe the server sends some absolutely unrelated content which SVNKit incorrectly treats as chunked stream. I would suggest to run this a couple of times to find out whether it fails reproducibly at the same moment.
If you’re using SVNKit programmatically, would you have an ability to debug it a bit? I would suggest to set a breakpoint at ChunkedInputStream::getChunkSizeFromInputStream and see what data really comes from the socket. Maybe this is not needed if we could get the traffic recorded.
If possible attach the results or if not then maybe your insights, why SVNKit can’t read the chunk size.
Normally, the chunk size is just a hexadecimal number followed by CRLF, so there’s not much room for something that could go wrong.
Thanks a lot. Here is a snippet of the SVNKit log right before it throws the exception. When I was debugging it, it seems got the Hex number of the size, but when it read through, it crapped out.
==========================================
Sep 20, 2022 11:09:18 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
FINEST: SENT
<?xml version="1.0" encoding="UTF-8"?>
<S:update-report xmlns:S=“svn:” send-all=“true”>
<S:include-props>yes</S:include-props>
<S:src-path>https://visualsvn.mususa.com/svn/FOS_US-ETD</S:src-path>
<S:target-revision>9944</S:target-revision>
<S:update-target>trunk/mbs-apps/mbs-portal</S:update-target>
<S:depth>infinity</S:depth>
<S:entry start-empty=“true” rev=“9944” depth=“infinity”></S:entry>
</S:update-report>
Sep 20, 2022 11:09:18 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
FINE: DEFAULT: socket output stream requested…
Sep 20, 2022 11:09:18 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
FINEST: READ
HTTP/1.1 200 OK
Date: Tue, 20 Sep 2022 15:09:18 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: default-src ‘none’
Vary: Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=5
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/xml; charset=“utf-8”
Sep 20, 2022 11:09:18 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
FINEST: READ
<?xml version="1.0" encoding="utf-8"?>
<S:update-report xmlns:S=“svn:” xmlns:V=“http://subversion.tigris.org/xmlns/dav/” xmlns:D=“DAV:” send-all=“true” inline-props=“true”>
<S:target-revision rev=“9944”/>
<S:open-directory rev=“9944”>
<S:open-directory name=“mbs-portal” rev=“9944”>
<D:checked-in><D:href>/svn/FOS_US-ETD/!svn/ver/9944/mbs-portal</D:href></D:checked-in>
<S:set-prop name=“svn:entry:committed-rev”>8722</S:set-prop>
<S:set-prop name=“svn:entry:committed-date”>2020-07-02T23:07:20.588727Z</S:set-prop>
<S:set-prop name=“svn:entry:last-author”>u390191</S:set-prop>
<S:set-prop name=“svn:entry:uuid”>72d0f78b-2473-4dd3-821f-e69e2d42af2e</S:set-prop>
<S:set-prop name=“svn:ignore”>.project
.classpath
.settings
.tern-project
node_modules
target
</S:set-prop>
Sep 20, 2022 11:09:18 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
FINE: NETWORK: chunked stream ended unexpectedly
java.io.IOException: chunked stream ended unexpectedly
at org.tmatesoft.svn.core.internal.util.ChunkedInputStream.getChunkSizeFromInputStream(ChunkedInputStream.java:113)
at org.tmatesoft.svn.core.internal.util.ChunkedInputStream.nextChunk(ChunkedInputStream.java:97)
Thanks for the log. It seems it doesn’t contain such low level information as chunk size. But when you were debugging, what was the input? Normally it should be
SIZE_IN_HEX\r\n
As I understand, it failed because “\r\n”, i.e. CRLF had been never sent as the protocol requires. But according to you, the hexadecimal digits were sent. What was the data after these digits and what these digits were?
Was the input maybe like:
SIZE_IN_HEX\n
? SVNKit doesn’t expect such an input, if this is what the server sends, this would explain the exception.
Appreciate the time and help. When debugging it, the chunk size HEX number seems to be parsed correctly - it is 31E. Looks like it fails on the subsequent reads as at some point, it gets to here
if (b == -1) {
throw new IOException(“chunked stream ended unexpectedly”);
}
Looks like the response is compressed (GZIP), would it be helpful to disable the GZIP response from server side?
Yes, it would be very helpful! And also it would be helpful to catch the traffic using Wireshark or some logging TCP Proxy (until recently, I’ve been using TcpCatcher succesfully but it stopped working for some reason).
b == -1 means the end of the input, so for now it looks like the input is just
31E
without CRLF. This would be strange and unexpected but Wireshark on plain (not gzipped) traffic could confirm that or refute that.