4

When using strip on certain object files in Amazon Linux, it errors with: $ sudo strip dfitpack.so BFD: st6qqyd2: Not enough room for program headers, try linking with -N strip:st6qqyd2[.note.gnu.build-id]: Bad value

The object files are created by pip install, packages including numpy, sklearn, pandas, xgboost, numexpr. What we are trying to do is strip down the code and object files to fit within an AWS Lambda 250MB limit.

A google search finds references to binutils on GNU mailing lists, but most tips are intended for developers building their own code, and to making changes in header files and the like. I suspect there is a build configuration change I can make when running pip install, and am hoping that someone with a better understanding of binutils can help.

Specs: $ uname -a Linux ... 4.4.5-15.26.amzn1.x86_64 #1 SMP Wed Mar 16 17:15:34 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $ yum info binutils ... Version : 2.23.52.0.1 Release : 55.65.amzn1 ...

Monica For CEO
  • 320
  • 1
  • 17
Emmet Murphy
  • 43
  • 3

1 Answers1

2

Some pip packages (or their dependencies) download binaries instead of downloading sources and building on the local machine, and there can be issues when running strip on a binary built in a different environment. To download sources and build on the local machine, try:

CFLAGS='-O0' pip install --no-binary :all: PACKAGE

See pip documentation at https://pip.pypa.io/en/stable/reference/pip_install/#install-no-binary

There was a discussion of this issue in the context of building Debian packages at https://github.com/spotify/dh-virtualenv/issues/150

Note that some binaries might not build with optimization flags, hence setting CFLAGS to disable optimization flags, see: https://stackoverflow.com/questions/16149613/installing-lxml-with-pip-in-virtualenv-ubuntu-12-10-error-command-gcc-failed

Also note that it can take a long time to build all dependencies from source, so if you have the time to sort through which packages have issues, and which do not, and you want to speed up the build, you could try some variation of this recipe to install only some packages from source and with varying levels of optimization:

CFLAGS='-O0' pip install --no-binary :all: PACKAGE
pip install --no-binary :all: PACKAGE
pip install PACKAGE
Community
  • 1
Monica For CEO
  • 320
  • 1
  • 17