Pages: Welcome | Projects

Git push speedup hack

2015/4/2
Tags: [ git ]

If you are using git together with a remote bare repository (classic case of GitHub, but you could do it on wherever server), chances are you are using SSH for pushing on the remote repo. It is quite convenient, as you can register your SSH public key on the server, and if you didn't specify a passphrase for it, you can speed up push and fetch operations by not having to type in any password. (As a side note, I guess the same would be feasible with HTTPS by using client-side SSL certificates, but they are not very popular).

SSH connection sharing

What people usually don't know is that SSH is a real piece of cake. It gives you lots of awesome hacky features. One I definitely love is the Master Connection, which allows a user to established a single connection with the server, and have it shared among all the ssh sessions.

Quoting the manual, on the -M option:

Places the ssh client into “master” mode for connection sharing. Multiple -M options places ssh into “master” mode with confirmation required before slave connections are accepted. Refer to the description of ControlMaster in ssh_config(5) for details.)

But you don't need to set the -M option every time: you can do it on the configuration file. So, this is my .ssh/config:

ControlMaster auto
ControlPath /tmp/dacav/ssh-%u@%l-%r@%h:%p
ServerAliveInterval=60
...
Host github.com
    User git
    IdentityFile /mnt/secure/dacav/ssh-prvkeys/github-id_rsa
...

Interesting points here:

This configuration by itself allows me to speed up the ssh connection to any server I'm already connected with. This happens because all the connection establishment and identity verification is already done for the Master Connection.

Using it with a remote Git repository

This is enough to establish any kind of connection with the remote server and keep it open. Any git push and git fetch command towards a server accessed through SSH will be faster as a consequence. But how about Github? Even though it optionally uses SSH as transport for git, they were not fool enough to give everybody a shell access.

Indeed you can try and fail to connect to github using SSH. You will get something like this:

$ ssh git@github.com
PTY allocation request failed on channel 1
Hi $user! You've successfully authenticated, but GitHub does not provide
shell access.
Connection to github.com closed.

But if you read your ssh (1) manpage you may find some solution to this problem, like the -N option, which corresponds to:

Do not execute a remote command. This is useful for just forwarding ports (protocol version 2 only).

So if we reapeat the experiment...

$ ssh git@github.com -N
...

The shell will hang, and you will maintain the connection active until you terminate it (e.g. with SIGINT aka CTR+C or SIGTERM).

Clever girl...

If you know your manual, at this point you might feel clever, and think about some other SSH options, like -D. This option creates a SOCKS5 on the specified local port, and every connection made to it will be tunneled on the server, and created server side. This is a nice trick to bypass censorship and/or restrictions.

If it works, we would be able to do HTTP requests from an IP address which belongs to Github.

So, from one terminal let's open the SSH tunnel:

$ ssh -D 127.0.0.1:8080 git@github.com -N

Which will open our socks proxy:

$ netstat -ltpn
...
tcp        0      0 127.0.0.1:8080          0.0.0.0:* LISTEN      19527/ssh           

$ nmap -sV localhost -p 8080
...
PORT     STATE SERVICE VERSION
8080/tcp open  socks5  (No authentication; connection failed)

(Darn', nmap! Spoiler alert!)

So we know it won't work, but let's try it anyway with curl:

$ curl --socks5 127.0.0.1:8080 http://roundhousecode.com

From your SSH shell you will see this:

$ ssh -D 127.0.0.1:8080 git@github.com -N
channel 2: open failed: administratively prohibited: 

While curl will (of course) fail.

$ curl --socks5 127.0.0.1:8080 http://roundhousecode.com
curl: (7) Failed to receive SOCKS5 connect request ack.

Tl;dr

You can speed-up your Git operation using SSH connection sharing, as you would do for regular SSH operations. But you are not allowed to do magic, what did you expect?

Also you should check out the manuals for cool options.