Backup upgradable packages with dpkg-repack on debian

Question

I want to list upgradable packages and automatically repack them to .debs. My try:

fakeroot -u dpkg-repack `apt list --upgradable | cut -f1 -d"/" | awk '{if(NR>2)print}'`

So it gets package names, then redirects names to dpkg-repack. It works partially, because dpkg-repack throws error when amd64 and i386 package exists and both have the same name - in this case it expects architecture to be added to package name.

Do you have any idea how to manage it and in case of multiple arch exists, auto generate both debs? It seems dpkg-repack is not smart enough to be able to generate multiple architectures automatically, it only throws error message saying multiple packages with the same name are installed

Answer 1

I use the following script to dpkg-repack all packages that would be affected by an apt-get operation:

#!/bin/zsh
#
# creates subdir and dpkg-repacks all relevant packages in it
#
DIR=$(date +%Y%m%d)
OPER=${@:-dist-upgrade}
mkdir -p $DIR
cd $DIR || exit 1
zmodload zsh/system

renice -n 20 -p $$ >/dev/null 2>/dev/null

nproc=$(nproc)
native_arch=$(dpkg --print-architecture)
typeset -U possible_arch=($native_arch all)
[[ $native_arch = amd64 ]] && possible_arch=($possible_arch i386)

function repacked() {
    local p=$1
    local a=$2
    local existing
    existing=(${p}_*${a}.deb(N) ${p}_*all.deb(N))   # no need to try repacking in a binary arch if _all already exists, because arch-specific and arch=all packages of the same name can't exist simultaneously
    if (($#existing)); then
        [[ "$a" = none ]] && unset a    # avoid confusing message with arch specified as "none"
        echo "Skipping $p${a:+:$a}, already repacked." >&2
        return 0
    fi
    return 1
}

function worker() {
    local pkg=$1
    local arch=$2
    local lockfd lockfd2
    local success=0
    local try_arch
    [[ -z "$pkg" ]] && return 0
    : >>"$pkg.$arch.lock"
    if zsystem flock -f lockfd -t 0 "$pkg.$arch.lock" 2>/dev/null; then
        if [[ $arch = none ]]; then
            for try_arch in $possible_arch; do
                repacked $pkg $try_arch && success=1
            done
            if ! ((success)) && ! dpkg-repack $pkg; then
                for try_arch in $possible_arch; do
                    echo "*** DEBUG: *** entering architecture guessing branch in worker()." >&2    # it's unclear whether this branch will ever even be run
                    repacked $pkg $try_arch && break    # avoid even locking if another thread got here first; allows main thread to start an additional worker sooner
                    : >>"$pkg.$try_arch.lock"
                    if zsystem flock -f lockfd2 -t 0 "$pkg.$try_arch.lock" 2>/dev/null; then
                        if repacked $pkg $try_arch; then    # make sure another thread didn't get to this package after our previous check but before acquiring the lock
                            success=1
                        else
                            dpkg-repack --arch=$try_arch $pkg:$try_arch && success=1
                        fi
                        rm $pkg.$try_arch.lock
                        zsystem flock -u $lockfd2
                        ((success)) && break
                    fi
                done
            fi
        else
            repacked $pkg $arch || dpkg-repack --arch=$arch $pkg:$arch  # TODO: run with loadwatch
        fi
        rm $pkg.$arch.lock
        zsystem flock -u $lockfd
    fi
}

# clean up any leftover locks from a previous invocation
for i in *.lock(N); do
    zsystem flock -f lockfd $i  # if they're still locked, a dpkg-repack may still be running; wait for it to finish
    rm $i
    zsystem flock -u $lockfd
done

for i in $(echo n \
    | LC_MESSAGES=C apt-get -d -u ${=OPER} 2>&1 \
    | sed -r '/^The following packages were automatically installed/,/^Use .apt(-get)? autoremove. to remove them/d
          /^The following NEW packages will be installed:/,/^The/s/^[[:space:]].*//
          /^The following packages have been kept back:/,/^The following packages will be upgraded:/d
          /^The following packages have unmet dependencies:/,$d' \
    | grep '^[[:space:]]' \
    | tr -d '*'); do
        if ((${i[(I):]})); then # package name includes architecture (separated by colon) -- index of colon within $i is not zero
            echo ${${i:t}/:/ }  # print name and architecture
        else
            p=${i:t}
            # guess architecture:
            [[ -e /var/lib/dpkg/info/${p}.list ]] && echo $p none   # we don't know the architecture, but perhaps dpkg-repack doesn't need it; have the worker try
            for try_arch in $possible_arch; do
                [[ -e /var/lib/dpkg/info/${p}:${try_arch}.list ]] && echo $p $try_arch
            done
        fi
done | sort -u | while read p a; do
    if ! repacked $p $a; then
        workers=(*.lock(N))
        while [[ $#workers -ge $nproc ]]; do    # wait for a worker slot to become available
            sleep 0.5
            workers=(*.lock(N))
        done
        [[ -n "$p" ]] && worker $p $a &
    fi
done
wait
rm -f *.lock(N)

Just call it with the same arguments you would pass to apt-get. It'll create a directory named for today's date, and put all generated .debs in there. It will repack as many packages in parallel as you have CPU cores. It's idempotent in the sense that if a .deb for a package already exists, it won't repack it again, making it safe to abort and restart.

It has the i386 and amd64 architectures hardwired, so if you have something else you'll need to change it.