Google Authenticator uses the TOTP algorithm to generate your One-Time Password (OTP). TOTP works like this : The server generates a secret key and shares with the client (you) when the client registers with the server. Using the shared key and the current timestamp, a new password is generated every 30 seconds.
If anyone has the shared key, then they can generate the OTP themselves using the TOTP algorithm. Isn't this similar to a password? Doesn't it get reduced to having two passwords - One is the password that you use to login and the other is the shared key between you and the server?