Can't upgrade certbot because of libc6 version

Question

I am trying to update Certbot from 0.10.2 to some version with support to ACME 2 in Debian 9.1 with Python 2.7.9.

apt-cache policy certbot
certbot:
  Installed: 0.10.2-1
  Candidate: 0.28.0-1~deb9u2
  Version table:
     0.28.0-1~deb9u2 500
        500 http://deb.debian.org/debian oldstable/main amd64 Packages
        500 http://deb.debian.org/debian oldstable-updates/main amd64 Packages
     0.28.0-1~bpo9+1 100
        100 http://ftp.debian.org/debian stretch-backports/main amd64 Packages
 *** 0.10.2-1 100
        100 /var/lib/dpkg/status

From here, this is what I am trying:

echo "deb http://ftp.debian.org/debian stretch-backports main">>/etc/apt/sources.list
apt-get update
apt-get install python-certbot-nginx -t stretch-backports

/etc/apt/sources.list

deb http://deb.debian.org/debian/ oldstable main contrib non-free
deb-src http://deb.debian.org/debian/ oldstable main contrib non-free

deb http://deb.debian.org/debian/ oldstable-updates main contrib non-free
deb-src http://deb.debian.org/debian/ oldstable-updates main contrib non-free

deb http://deb.debian.org/debian-security oldstable/updates main
deb-src http://deb.debian.org/debian-security oldstable/updates main

deb http://ftp.debian.org/debian stretch-backports main
deb-src http://ftp.debian.org/debian stretch-backports main

||/ Name                                Version                Architecture           Description
+++-===================================-======================-======================-============================================================================
ii  libc6:amd64                         2.24-11+deb9u1         amd64                  GNU C Library: Shared libraries

ERROR:

apt-get install python-certbot-nginx -t stretch-backports
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libc-dev-bin : Depends: libc6 (> 2.28) but 2.24-11+deb9u1 is to be installed
                Recommends: manpages-dev but it is not going to be installed
 libc6-dev : Depends: libc6 (= 2.28-10) but 2.24-11+deb9u1 is to be installed
 locales : Depends: libc-bin (> 2.28) but 2.24-11+deb9u1 is to be installed
 python-certbot-nginx : Depends: python3-certbot-nginx but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

libc6:
Installed: 2.24-11+deb9u1
  Candidate: 2.24-11+deb9u4
  Version table:
     2.24-11+deb9u4 500
        500 http://deb.debian.org/debian oldstable/main amd64 Packages
 *** 2.24-11+deb9u1 500
        500 http://deb.debian.org/debian-security oldstable/updates/main amd64 Packages
        100 /var/lib/dpkg/status
libc6-dev:
  Installed: 2.28-10
  Candidate: 2.28-10
  Version table:
 *** 2.28-10 100
        100 /var/lib/dpkg/status
     2.24-11+deb9u4 500
        500 http://deb.debian.org/debian oldstable/main amd64 Packages
     2.24-11+deb9u1 500
        500 http://deb.debian.org/debian-security oldstable/updates/main amd64 Packages
locales:
  Installed: 2.28-10
  Candidate: 2.28-10
  Version table:
 *** 2.28-10 100
        100 /var/lib/dpkg/status
     2.24-11+deb9u4 500
        500 http://deb.debian.org/debian oldstable/main amd64 Packages
     2.24-11+deb9u1 500
        500 http://deb.debian.org/debian-security oldstable/updates/main amd64 Packages
libc-dev-bin:
  Installed: 2.28-10
  Candidate: 2.28-10
  Version table:
 *** 2.28-10 100
        100 /var/lib/dpkg/status
     2.24-11+deb9u4 500
        500 http://deb.debian.org/debian oldstable/main amd64 Packages
     2.24-11+deb9u1 500
        500 http://deb.debian.org/debian-security oldstable/updates/main amd64 Packages

Any idea? This is a production server.

EDIT:

So apparently I have installed some packages from the Debian 10 in Debian 9. Since I can't do anything with apt, I am trying to solve the dependencies with aptitude.

(venv)root@deb64:/home/x/# aptitude -V -s install libc6-dev=2.24-11+deb9u4
The following packages will be DOWNGRADED:
  libc6-dev{b} [2.28-10 -> 2.24-11+deb9u4]
The following packages will be REMOVED:
  libc-dev-bin{u} [2.28-10]
The following packages will be upgraded:
  libc6 [2.24-11+deb9u1 -> 2.24-11+deb9u4]  linux-libc-dev [4.9.30-2+deb9u5 -> 4.9.210-1]
The following partially installed packages will be configured:
  libc-l10n  locales{b}  man-db
2 packages upgraded, 0 newly installed, 1 downgraded, 1 to remove and 235 not upgraded.
Need to get 6,539 kB of archives. After unpacking 3,561 kB will be freed.
The following packages have unmet dependencies:
 libc6-dev : Depends: libc-dev-bin (= 2.24-11+deb9u4) but it is not going to be installed
 locales : Depends: libc-bin (> 2.28) but 2.24-11+deb9u1 is installed and it is kept back
The following actions will resolve these dependencies:

     Remove the following packages:
1)     locales [2.28-10 (now)]
2)     task-english [3.39 (now, oldstable)]

     Install the following packages:
3)     locales-all [2.24-11+deb9u4 (oldstable)]

     Upgrade the following packages:
4)     postgresql-9.4 [9.4.13-0+deb8u1 (now) -> 9.4.26-2.pgdg90+1 (stretch-pgdg)]
5)     postgresql-9.6 [9.6.4-0+deb9u1 (now) -> 9.6.18-1.pgdg90+1 (stretch-pgdg)]
6)     postgresql-contrib-9.4 [9.4.13-0+deb8u1 (now) -> 9.4.26-2.pgdg90+1 (stretch-pgdg)]
7)     postgresql-contrib-9.6 [9.6.4-0+deb9u1 (now) -> 9.6.18-1.pgdg90+1 (stretch-pgdg)]

     Downgrade the following packages:
8)     libc-dev-bin [2.28-10 (now) -> 2.24-11+deb9u4 (oldstable)]

Accept this solution? [Y/n/q/?] q

It's safe to the system if I accept this solution?

ii  libc6:amd64                       2.24-11+deb9u1                 amd64        GNU C Library: Shared libraries
iU  libc6-dev:amd64                   2.28-10                        amd64        GNU C Library: Development Libraries and Header Files

Why not to backup and try? With CoW FSes like, say, Btrfs you could just snapshot the system and give it a go. You have similar options with any FS _and_ LVM-2, although it's somewhat different. Also, you might see why people prefer to run more and more apps in own isolated environments — be it a container based (say, Docker or LXC, _etc_) or VMs. — poige, Jun 02 '20 at 02:46
@poige this server is a VM with recurring snapshots. The problem here is the downtime if the upgrade goes wrong. — Fel, Jun 03 '20 at 16:28
Do you or did you have other package sources listed (`/etc/apt/sources.list` or `/etc/apt/sources.list.d/*`)? You clearly have some libc6 2.28-10 stuff (matches the version in stable/buster, rather than oldstable/stretch) from somewhere. You now have `oldstable` in the sources.list file, was perhaps something accidentally installed with it saying `stable` after buster came out or something like that? — Håkan Lindqvist, Jun 06 '20 at 18:07

score 3 · Answer 1 · answered Jun 03 '20 at 02:57

I would not recommend mixing and matching packages meant for different major versions of a distro because sometimes dependencies are pulled in that destablize the whole. If at all possible I would "rebuild" the server elsewhere where the packages sources haven't mixed because libc is a major thing to have changed...

As others have suggested, is it possible for you to run your services inside something like docker so each can have the environment it wants without polluting that of the host? That way if things go sideways you can tear down the container and build a new one...

TLDR; I'm worried for your server because it sounds like you have "crossed the streams".

score 2 · Accepted Answer · answered Jun 07 '20 at 01:03

2

Finally got a solution:

apt install libc6/stretch libc6-dev/stretch libc-dev-bin/stretch libc-bin/stretch locales/stretch linux-libc-dev/stretch

Another user have the same problem and fortunately someone find the solution here.

answered Jun 07 '20 at 01:03

Fel

1,432
1
10
17

I'd accept your own answer to recoup some your bounty points... – Anon Jun 07 '20 at 06:23
1

@Anon From [here](https://stackoverflow.com/help/bounty): "You cannot award a bounty to your own answer" – 9072997 Jun 07 '20 at 21:42

hargut · Answer 3 · 2020-06-04T21:25:09.803

0

The first view of your config looks good as:

you are running Debian 9 which is stretch
oldstable currently points to stretch

All of the listed configs show values which should work together. As /var/lib/dpkg/status lists for some packages versions that are newer than the ones that are listed it has to be assumed that there was at some point:

other repositories listed as shown in the configs
packages had been manually installed

As libc6 is involved, caution/attention is needed for any single step. If this package is broken/removed the system will be in an "inoperable" state.

Personally I'd suggest to stick with one tool to fix the situation, and would use apt and dpkg only.

What does apt-get install -f suggest to be performed?
Removing the python-certbot-nginx package using dpkg likely will bring back apt functionality.
Once apt is operating again it would be needed to downgrade at least libc6-dev and libc6-dev-bin, and ideally all other packages that had been pulled in from the "other" repository.

After this situation is cleared, a way forward can be to manually build a backport of the python-certbot-nginx package by doing the following:

Add a deb-src entry for Debian 10 and update.
apt-get build-dep python-certbot-nginx which installs all required build dependencies for certbot
apt-get source python-certbot-nginx -b fetches the source package from Debian 10 and attempts to build it with the previously installed dependencies from Debian 9
Install the resulting.deb file using dpkg

edited Jun 04 '20 at 21:25

answered Jun 04 '20 at 21:05

hargut

3,848
6
10

ignoring request to remove python-certbot-nginx which isn't installed. apt-get install -f seems dangerous. A lot of things to remove – Fel Jun 06 '20 at 01:10
How does the output of `dpkg -l | grep libc6` look like? – hargut Jun 06 '20 at 11:52
What does `dpkg -l | grep -v ^ii` return? – hargut Jun 06 '20 at 11:59
https://justpaste.it/7i8ca and question updated – Fel Jun 06 '20 at 17:58
The packages listed with `it` and `iU` are the ones causing the issue. The `rc` marked items can be ignored, as these are already removed. As you already re-installed them with the correct version the system should be ok now, except eventually successfully installed packages might still be around. – hargut Jun 07 '20 at 11:28
Did the manual backport approach for the new certbot version work out? – hargut Jun 07 '20 at 11:29
1

Yes,, it works. – Fel Jun 08 '20 at 14:56

Can't upgrade certbot because of libc6 version

3 Answers3