No. Analog telephone modems (a.k.a. POTS - "Plain Old Telephone Service" modems) worked because when the traditional telephone network (a.k.a the PSTN - "Public Switched Telephone Network") digitized the audio, it did so with 8-bit samples 8,000 times per second, for a total of 8 bits * 8 kHz = 64,000 bps digital audio. It wasn't possible to fully utilize that full 64kbps for data because of problems with the analog <-> digital conversion (ADC/DAC), but if you had a digital modem on a digital phone line (like an ISDN B-channel), you could send at 53-56kbps through a single DAC. But from the analog line, you could only send 33.8kbps through a single ADC.
But the problem with cell phones is that they've never provided a full 64kbps digital audio channel like the landline PSTN did. They use lossy codecs to compress your voice audio down to fit into a 9600 bps channel. So if you want to try to do audio-modulated signaling over that channel, I think you'd be lucky to get 4800 bps (4.8kbps = 0.6 KibiBytes/sec) out of it. And that's assuming you design custom modulation schemes to get the most out of what that audio channel gives you. I have no idea if any of the traditional modulation schemes used by POTS modems would work well over GSM voice channels. For all I know, an off the shelf POTS modem may just completely fall on its face and not even be able to sustain the ages-old 300 bps rate over a GSM voice channel.
Update: I found a reference online that said that doing analog audio modulation over GSM voice channels was typically limited to 2400 bps, so my estimate above was quite generous.
This goes a little outside the scope of your question, but note that technologies existed to make dialup-like connections over digital cellular networks but without the severely rate-limiting analog complication of hooking a POTS modem to the headset jack of the cell phone.
One solution was called "Circuit-Switched Data" or CSD, and usually involved hooking your computer to your cell phone via a serial connection of some type (using traditional RS-232 or RS-422 style serial interfaces or IrDA in the old days, and USB or Bluetooth more recently), and having your cell phone give that serial interface direct access to the 9600 bps data connection to the wireless carrier's equipment. Inside the wireless carrier's network, your 9600 bps data stream would be fed into a digital modem hooked up to a digital phone line, so you could use this technology to make a 9600 bps modem call to any modem on the PSTN.
CSD was followed by High-Speed CSD (HSCSD), which was like CSD except your cell phone would use 4 GSM voice channels simultaneously, so you'd be able to make ~56kbps connections, just like (in the downstream channel at least) the last era of POTS modems (V.90).
Over digital cellular technologies like GPRS and EDGE, it was possible to use that serial connection to the phone to do PPP, but that wouldn't have worked to just any modem hooked up to any phone line; the other end of your PPP connection would need to be a PPP server that receives its PPP connections not from modems, but over IP, tunneled in L2TP. This was common among ISPs that outsourced their dial-up modem POPs to third parties like Level 3 and UUNet, and is still common today among DSL ISPs (that's why so many DSL services require PPPoA or PPPoE).
tl;dr: To answer your specific question, No, it is probably not possible to hook a POTS modem via headset jack to a cell phone and make a successful connection to a modem on the PSTN. If it connects at all, it will probably only use extremely low data rates (even as POTS modems go), and be glitchy and unreliable. However, there are (or have been) other ways to connect a computer to a cell phone to make dial-up modem calls or PPP connections.
6Most phones actually have a serial port accessible over USB that has the same interface as a serial modem and long ago this was used with laptops to access the internet via dial up. – Dan D. – 2014-04-30T20:06:04.670
4Someone can call me out on this, but I'm fairly sure the data being send over dial up connection is more than just audio. – Moses – 2014-04-30T20:11:47.207
1@Moses How do you mean? The data is converted to analogue so it can be sent over a connection that is normally used for just audio. – SilverlightFox – 2014-04-30T20:15:18.373
6@Moses POTs can only transmit audio. – Elliott Frisch – 2014-04-30T20:17:02.377
1@DanD. Didn't these phones still go onto a dial-up modem? As Moses said, I think more than just audio is sent over the line. The real question is why are you trying to use new tech to enable really, really old tech? Surely there's a better tethering solution than aiming for dial-up. – Wutnaut – 2014-04-30T20:17:25.317
3It is a good technical question, I think, but why do you need it today? Is it only way your computer can be connected to Internet? – VL-80 – 2014-04-30T20:22:02.487
6Tethering would make more sense. – Joseph – 2014-04-30T20:24:57.300
1The purpose of this is basically for emergency use or when there's no wifi/4G signals available. I also don't have a tethering plan for my phone. So that's why I'd like to be able to access the internet anytime and anywhere I want, even if it's dialup through my phone. Usually everywhere I go still has phone service, but not necessarily internet. I also have an unlimited calling plan. – Generalkidd – 2014-05-01T17:20:54.173
Well I'm actually using a Nokia Lumia 928 Windows Phone so I can't really root it to tether. There is a tethering app available on Windows Phone but it has its limits and sometimes in the city the network could get congested. – Generalkidd – 2014-05-02T11:44:23.083