How do I automate git pulls from a private repository using Github's hooks?

Question

I'm sure there's plenty of similar questions to this, but mine's a little more specific. I have a server used for testing, and the site owner wants it to automatically update itself when someone pushes to the repositories. I figured I'd use the Github hooks to send some data to a specific URL, signaling that it's time for an update. However, our repos are private, and if I run a shell_exec() in PHP, it runs it as www-data, which can't have ssh keys. I don't want to put passwords in cleartext, and I'm pretty stumped as to how I'm going to allow the user to authenticate. Can anyone provide some advice? I've been pulling out my hair for too long over something that seems like it should be pretty simple.

Answer 1

The reason why the git pull step fails is because on Ubuntu, Apache executes the script as the user www-data. Thus, git looks for the ssh keys associated with the user www-data and failing to find them, is unable to complete the git pull request.

On Ubuntu 16.04, the user www-data is assigned the home directory /var/www. This is the directory in which git looks for the ssh keys to negotiate the transfer. Thus the solution is make GitHub believe that the user www-data is real by assigning it a valid set of keys. To break down the steps:

Note: This assumes that you have sudo access.

1. Create a directory /var/www/.ssh owned by www-data:www-data

   $ sudo mkdir -p /var/www/.ssh
   $ sudo chown -R www-data:www-data /var/www/.ssh
   

2. Create ssh keys in the directory

   $ cd /var/www/.ssh
   $ sudo ssh-keygen -t rsa -b 2048
   

   When ssh-keygen asks for the directory to put the keys in, choose /var/www/id_rsa

3. Ensure that the persmissions and ownership of the keys is correct. chown to www-data:www-data as necessary.

   $ ls -la /var/www/.ssh/
   total 24K
   drwxr-xr-x 2 www-data www-data 4.0K Apr 29 23:58 ./
   drwxr-xr-x 5 root     root     4.0K Apr 30 00:06 ../
   -rw------- 1 www-data www-data 1.7K Apr 29 23:33 id_rsa
   -rw-r--r-- 1 www-data www-data  394 Apr 29 23:33 id_rsa.pub
   

4. Copy the id_rsa.pub key to the authorized ssh keys in the GitHub repository settings.

5. It is important to ensure that the git pull works when performed as the user www-data. Using ssh also needs adding the GitHub server identity to the known_hosts file. However, the user www-data does not have a login shell by default. So we have to use a simple trick:

   $ sudo vi /etc/passwd
   

   Find the line for www-data and change the /usr/sbin/nologin to /bin/bash and save the file. The entry for www-data should look similar to:

   www-data:x:33:33:www-data:/var/www:/bin/bash
   

6. Change to the user www-data

   $ sudo su
   # su - www-data
   

7. Once you are logged in as www-data, go to the git repository and perform a git pull manually.

8. The ssh process will ask you to add the identity of the GitHub server to known_hosts file and use the key pair under /var/www/.ssh to complete the git pull.

9. If it succeeds, you should be set. Try to push a commit to GitHub from another computer and verify that the PHP script executes the pull request.

10. Reset the /etc/passwd file to it's original state with the login shell of the www-data user as /usr/sbin/nologin

Answer 2

There are many ways. Here's one:

1. This method only requires very basic authentication, so generate a UUID to use as an authentication token. For example: https://www.uuidgenerator.net/version4
2. Configure the GitHub repo's webhook to trigger your webhook, adding the UUID as a token: http://example.com/webhook.php?token=yourUUID

3. Put a webhook.php file in your site's root that checks the token and does nothing but flag the site for an update:

   <?php
   $token = "yourUUID";
   if($_GET['token'] != $token) {
       die("Unauthorized source!");
   } else {
       touch(__DIR__ . '/git_pull_needed');
   }
   

4. Create a cronjob that runs often (every minute?) and exits if the git_pull_needed file doesn't exist. If it does exist, the job can perform a git pull after which it deletes the git_pull_needed file. Note: The user this cronjob runs as can be root or preferably some 'deploy'-user that has an SSH key with read-only permission on GitHub and write permission to the webroot.

The main benefits of this decoupled method are the low risk of exposing the webhook.php to the outside (an attacker can only trigger a git pull if he somehow guesses the token), the disconnect between the webserver-user and the 'user' holding the ssh-key and the built-in simple rate-limiting.