Will remove a package and all configuration files.
apt-get --purge remove package
If it is still giving you issues, try reinstalling then purging:
apt-get --reinstall install package
apt-get --purge remove package
You still might need to:
killall package
Reboot and if it started again run:
update-rc.d remove package
Story Time: Once for some unknown reason, when I purge removed ejabberd; it continued running. I trusted apt-get to kill it and remove it. It didn't. Needless to say, it eventually filled up my entire /var with a ramdisk log file. It continued until I killed all of the ejabberd processes and that quickly resolved the issue. So it taught me to always be sure that apt-get worked and killed all needed processes.