Can't push to git on local server (public key) auth and then disconnect

Question

I recently setup a local test server with Gitlab following this guide https://www.digitalocean.com/community/tutorials/how-to-set-up-gitlab-as-your-very-own-private-github-clone

I am trying to make my first commit using an admin account that I created in the Gitlab interface. I have added my public key, it is the only key I have generated on the client computer. This is my output when trying to push:

$ git push -u origin master
Access denied.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

And this is my auth.log

Sep 28 23:35:32 KSubuntu02 sshd[1124]: Accepted publickey for git from 10.2.0.26 port 54658 ssh2: RSA fd:a6:2b:48:5d:9d:f9:6d:1a:93:cf:6e:f5:93:28:ac
Sep 28 23:35:32 KSubuntu02 sshd[1124]: pam_unix(sshd:session): session opened for user git by (uid=0)
Sep 28 23:35:32 KSubuntu02 systemd-logind[5524]: New session 33 of user git.
Sep 28 23:35:32 KSubuntu02 sshd[1172]: Received disconnect from 10.2.0.26: 11: disconnected by user
Sep 28 23:35:32 KSubuntu02 sshd[1124]: pam_unix(sshd:session): session closed for user git

Being stuck in this situation, what are some things I can do to try to fix this issue? I have root access on both client (mac) and server (Ubuntu 14).