Error connection is not available request timed out during SVN to GIT migration

santhosh · June 30, 2021, 7:25am

hi Ildar,

This is the same auto-configuration restart issue that I was referring to. As a work around I started an import with manual configuration and it is currently in progress now.

I have two major questions at this point of time.

We are in discussions to purchase the plugin as our trail is coming to end closely. What kind of support that an enterprise (licensed) customers get from TMATE if there are any issues in future during our migration journey. Is it the same way that we are discussing right now , or do we have a separate support channel/communications for licensed customers ?
From the mirror logs I observed that for every 1000 revisions it is doing a git GC, which is going on almost 2 hours and only after 2 hours the sync process is getting re-started.

Is it normal ? 2 to 3 hours of GC for every 1000 revisions ?

gc started : 21:41
gc ended : 00:42

 4637 2021-06-28 21:41:04,214 sync - Running periodical Git garbage collector.
 4638 2021-06-28 21:41:04,270 sync - Executing [git, -c, gc.autoDetach=0, gc]; environmentVariables={};workingDirectory=/bitbucket_home/shared/data/repositories/724
 4639 2021-06-29 00:42:26,895 sync - Getting content of "" directory at revision 23433
 4640 2021-06-29 00:42:27,308 sync - SET_PATH '' 23434 not empty depth=infinity

gc started : 01:19
gc ended : 02:54

 8215 2021-06-29 01:19:15,061 sync - Running periodical Git garbage collector.
 8216 2021-06-29 01:19:15,068 sync - Executing [git, -c, gc.autoDetach=0, gc]; environmentVariables={};workingDirectory=/bitbucket_home/shared/data/repositories/724
 8217 2021-06-29 02:54:38,334 sync - SET_PATH '' 25060 not empty depth=infinity
 8218 2021-06-29 02:54:38,663 sync - SET_PATH 'trunk' 25059 not empty depth=infinity

ildar.khusainov · June 30, 2021, 9:24am

Hi Santosh,

there are no add-on options in the Atlassian Marketplace that would inculde a dedicated support so all the SVN Mirror licenses available throug the Marketplace offer only Basic support which only includes support through open spaces on this forum with response during 3 working days.
However, we also have an extended Dedicated Support contract available for Euro 9900/year. In scope of the contract, we offer a dedicated private space on the support forum not visible to anybody else but our support and users you add explicitly, email support channel, 4 hours of voice support per year and guarnteed response during one working day. This contract is being sold separately, not throug the Atlassian Marketplace.
We also offer per-case support either through the email or forum option or using voice/video conferencing, here aer little more details:

TMate SubGit: Support

As for the triggering GC every 1000 revisions – this is a default setting, so that is expected. The time needed for the garbage collection depends completely on the repository and what actions/changes were made in the commits, for some repositories and revisions 2 hours it’s pretty normal and expected. It’s too much for the import process, though, so I think it may worth to switch the GC off completely for the import or at least make it triggered less often. It can be done with the following setting:

[svn]
    …
    triggerGitGC = false

or

[svn]
    …
    triggerGitGC = <number of revisions>

This setting should be added right to the mapping configuration prior the initial import start. The former setting (false) switches GC completely, the latter tells the add-on to trigger GC every <number of revisions>.

santhosh · June 30, 2021, 9:38am

Thanks Ildar,

I will go through the support plan details.

Can I add the value “triggerGitGC = false” for the current import which is currently in-progress and apply changes ? Is that possible . Or if I add , will it restart the import process ?

At the same time , do we have any adverse affects of disabling GC

ildar.khusainov · June 30, 2021, 9:42am

The setting won’t be applied on the fly, it needs to be applied explicitly by the button and the import has to be restarted (stopped and started again, not restarted from the very beginning) to apply this settings.
A possible drawback is that the repository may take more disk space rather that in case when GC is enabled. However, it’s only for the import time, after the initial import finished the setting can be switched back of the GC can be triggered manually in the repository.

santhosh · July 2, 2021, 4:50am

hi Ildar,

We have got a new issue now, suddenly the import stopped at the revision : 46983 for the last 10+ hours.

I usually check the logs at this repo location , yesterday when I tried to check the log, to my surprise there is no log file svnmirror.log

I am periodically checking this log file and it is getting truncated but suddenly the log file got disappeared at all.

Here is the screenshot. There is no svnmirror.log . It disappeared

And the importing got stuck at this stage

I have extracted the last log file that got zipped and attaching it here for your reference.
svnmirror-trunc.log (1.3 MB)

ildar.khusainov · July 2, 2021, 5:17am

Hi Santosh,

unfortunately, the last log does not contain much information, just a process of translating some revision, not even clear which one. The process itself seems to run smooth, however.
The fact that the log has not been created may indicate that the process indeed stuck, so it worth try to stop the import with the Stop button and restart it a couple of minutes later.
Disappearing log is very strange issue since normally the add-on does not remove this file. Could you advise could any other process remove that file?

santhosh · July 12, 2021, 7:33am

hi Ildar,

Greetings.

We went ahead and purchsed the plugin . Can you check on this at the earliest.

But currently its loading, this is what we see in bitbucket settings page.

ildar.khusainov · July 12, 2021, 10:24am

Hello Santosh,

this error indicates the user interface (front-end application) has difficulties communicating with the backend. This often appears when some network communication problems are present, especially if the Bitbucket Server is being accessed over a VPN tunnel or over an SSH tunnel. Also, it may appear when there are problems using long polling mechanism, for example, if the web browser (or reverse proxy on the Bitbucket Server) does not support long polling. If any of the above may be present, it may worth trying to not to use long polling. To disable long polling de-select the “Use Long Polling” checkbox on the “Add-on settings” tab in global SVN Mirror setting:

santhosh · July 12, 2021, 11:08am

But Ildar,

In order to do the changes, I should be able to access the “SVN MIRROR” Settings right. Im not even able to access the same settings page.

ildar.khusainov · July 12, 2021, 11:28am

Hello Santosh,

I understand that, but I’m afraid there is no other way to change the setting but the settings in the UI. Do I understand correctly you are accessing the Bitbucket Server over a VPN tunnel? Could it be possible to do that from the local network where the server resides (at least to change the setting)? Also, have you tried to use another web-browser?

santhosh · July 12, 2021, 12:49pm

hi Ildar,

I have tried using

Microsoft edge
Safari
Google Chrome

Issue is still the same.

Is there a way we can change these settings from the server in any configuration file.
Why is this behaving this way suddenly ? It was working fine until Friday. Do you want to check any logs to understand this.
Does disable/enable (or) uninstall/install of the plugin will help ?
Did it happen with any customer before ?
As due to pandemic everybody is working remote , from what I understand everyone should be definately using company VPN right . I dont understand when you say local network . Do you mean should we be in the office location network which is hosting the bitbucket server ?

I’m kind of stuck and have 2 BIG repositories pending.

santhosh · July 12, 2021, 2:39pm

hi Ildar,

We are stuck and unable to use your plugin. Looks like a dead end for us. Can you help.

ildar.khusainov · July 12, 2021, 3:11pm

Hi Santosh,

sure, we were trying to find a way to switch the Long Polling off in your situation. A possible way to do that is to use SVN Mirror add-on’s REST API, here is the request that switched the setting off:

curl -u 'USER:PASSWORD' -H "Content-Type: application/json" -H "X-Atlassian-Token:no-check" --data '{"enabled" : false, "interval" : 60}' "https://BITBUCKET_SERVER_URL/rest/svn/1.0/config?name=longPollOptions"

Replace the USER, PASSWORD and BITBUCKET_SERVER_URL placeholders with correspondent user, password and the URL. If you don’t have cURL on your computer (which is usually true for Windows) use any other means to sent a POST request, for example, Postman software.

Also, I’d like to ask if it’s true that you have many web browser tabs or windows opened with SVN Mirror UI opened in those tabs? Such a situation may also disturb long polling operations and this may be the reason it does not work today while it worked last week.

santhosh · July 12, 2021, 3:37pm

hi Ildar,

No luck . (Masking url and passwords)

# curl -k -u 'abcd:abcd' -H "Content-Type: application/json" -H "X-Atlassian-Token:no-check" --data '{"enabled" : false, "interval" : 60}' "https://OUR_URL/rest/svn/1.0/config?name=longPollOptions"
#

After executing above CURL , this is what I see

FYI - Previous to executing this command

We have also uninstall/installed
We have also disabled/enabled

ildar.khusainov · July 12, 2021, 4:16pm

Hi Santosh,

could you please also add the -v cURL option, just to make sure the request went well:

curl -v -k -u…

What about the number of tabs, could it be the case you have too many of them opened? Also, could you please advise are Bitbucket Server pages available and work well and if only SVN Mirror add-on pages are affected?

santhosh · July 12, 2021, 4:54pm

Hi Ildar,

I have closed all other tabs and tried - no luck
Have asked my colleague from other location in India - same error for him .
Have asked my colleague from US - same error for him .
bitbucket works perfectly fine , only issue with this particular plugin page.

Here is the curl output with verbose option. (Masked sensitive infromation with **)

[root@]# curl -v -k -u 'USER:PASS' -H "Content-Type: application/json" -H "X-Atlassian-Token:no-check" --data '{"enabled" : false, "interval" : 60}' "https://url/rest/svn/1.0/config?name=longPollOptions"
* About to connect() to *** port 443 (#0)
*   Trying **...
* Connected to *** port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: CN=**,ST=CA,C=US
*       start date: Jul 13 18:06:19 2020 GMT
*       expire date: Jul 13 18:06:19 2023 GMT
*       common name: d**
*       issuer: **
* Server auth using Basic with user '**'
> POST /rest/svn/1.0/config?name=longPollOptions HTTP/1.1
> Authorization: Basic Q2hha2lTYTpDaGludHVANDM5
> User-Agent: curl/7.29.0
> Host: **
> Accept: */*
> Content-Type: application/json
> X-Atlassian-Token:no-check
> Content-Length: 36
>
* upload completely sent off: 36 out of 36 bytes
< HTTP/1.1 500
< Content-Type: application/json
< Vary: accept-encoding
< X-AREQUESTID: @66KSMYx586x240284x30
< X-AUSERID: 906
< X-AUSERNAME: **
< X-Content-Type-Options: nosniff
< Set-Cookie: ApplicationGatewayAffinity=e20670a2de0f64d827dc4697ec998195d0bf74ae97a76d4c4122e00ce9902ac7;Path=/;Domain=**
< Set-Cookie: ApplicationGatewayAffinityCORS=e20670a2de0f64d827dc4697ec998195d0bf74ae97a76d4c4122e00ce9902ac7;Path=/;Domain=**;SameSite=None;Secure
< Date: Mon, 12 Jul 2021 16:49:09 GMT
< Connection: close
< Content-Length: 0
<
* Closing connection 0
[root@]#

santhosh · July 13, 2021, 6:29am

hi Ildar,

Any update on this please ? This is what we still see.

We are kind of running out of schedule and this is important for us

ildar.khusainov · July 13, 2021, 6:53am

Hi Santosh,

we are investigating this right now, I’ll be keeping you informed.
Meanwhile, we could use the REST API for the mirroring operations, too, could you please advise what the changes or operations do you need to perform in repositories?

ildar.khusainov · July 13, 2021, 11:46am

Hi Santosh,

we continue investigating the issue, yet haven’t managed to re-produce it in our test environment. May I ask you to share Bitbucket and SVN Mirror logs with us to investigate? I’m afraid add-on’s Support ZIP feature is not much of a help currently, but it’s possible to collect it directly from the filesystem. Also, could you please advise what is your Bitbucket Server version and edition, how many node are in the cluster (if that is Datacenter cluster), and how did you install the add-on – have you installed it from Atlassian Marketplace or in some other way?

santhosh · July 13, 2021, 4:37pm

Hi Ildar,

We just want to import a project with gitgc as false in a specific bitbucket repo . This is what the REST API that we need for now.