Python PEX loading

Question

I've been trying to wrap my head around the python pex utility (https://pex.readthedocs.org/en/latest/) for bundling some applications into .pex files for deployment.

My app isn't large enough to require twitters pants build tool, as well as I have some build requirements which pants doesn't address. I did however, try the pants tools build system using python_binary which resulted in pex files with the sources loaded into the pex files root. The BUILD files in pants accept a sources property for python_binary which can be a glob for files within the directory the build is running in, however, pants is using the pex programmatic API and not the command-line utility.

The problem is when I use the pex command-line utility on it's own, it seems to want distributions (i.e. folders set up with a setup.py, etc...) and wants to install my code into the .deps folder in the pex file rather than just copying the python files into the root of the pex file like pants seems to do.

Is copying the files over (not installing a package) not possible through the command-line pex tool?

score 3 · Accepted Answer

As of pex 1.0.0 there is no facility to glob up files and directories directly, you must have a setup.py as you suggest, or use pants (no longer Twitter's by the way - independent).

So you have 3 paths forward (#1 you already know, but spelling it out for others):

Create a setup.py and point the pex tool at its dir

$ tree -h
.
├── [4.0K]  lib
│   ├── [   0]  __init__.py
│   ├── [  38]  lib.py
│   └── [  68]  main.py
└── [  76]  setup.py

1 directory, 4 files
$ cat lib/lib.py 
def func():
    return 'func in lib'

$ cat lib/main.py 
from .lib import func

if __name__ == '__main__':
    print(func())

$ cat setup.py 
from setuptools import setup

setup(
    name='lib',
    packages=['lib']
)

$ pex . -e lib.main -o lib.pex
$ ./lib.pex 
func in lib

NB: The . in the pex command line is the bit pointing pex at this dir's setup.py

File an issue against pex to support a set of files in place of a requirement / setup.py. You can do that here.
File issues against pants to support the requirements you have that it does not address. You can do that here

As a pants committer I can say that we're working towards making pants easier and easier to use such that no project is too small. You should be able to pip install pantsbuild.pants.backend.python && touch pants.ini and be up and running in a python-only repo but we're not there today.

Python PEX loading

1 Answer 1

自怼圈/年度番新