Pycon Japan

tarek@ziade.org / @tarek_ziade

お前、誰よ

Presenter Notes

Agenda / 概要

  • Packaging
  • Python 3

Presenter Notes

Packaging / パッケージング

Presenter Notes

Packaging drive people insane

hate

Presenter Notes

パッケージングで一般的な5つの課題

  1. How do I define dependencies ? / 依存関係をどう定義するか?
  2. what's setup.py role ? / setup.py の役割は何か?
  3. what's installed ? / 何がインストールされるか?
  4. what about versions schemes ? / バージョンをどう付けるか?
  5. how to define data files ? / データファイルをどう定義するか?

Presenter Notes

Dependencies / 依存関係

Distutils — module-level dependencies:

Done in setup.py:

1 from distutils.core import setup
2 setup(requires=['ldap'])

Then:

1 python setup.py sdist

⇒ Line "Requires: ldap" generated in PKG-INFO

PKG-INFO == Metadata v1.1

PKG-INFO installed in site-packages (Py>=2.5)

Presenter Notes

Dependencies / 依存関係

Setuptools — project-level dependencies:

1 from setuptools import setup
2 setup(install_requires=['lxml'])

"lxml" in requires.txt

requires.txt installed in site-packages

pkg_resources in setuptools to read it back.

Presenter Notes

Dependencies / 依存関係

Debian — system-level dependencies:

1 $ apt-get install libxslt-dev

"libxslt" installed on the OS.

Presenter Notes

Dependencies / 依存関係

  • Distutils — module-level dependencies

Gone !

  • Setuptools — project-level dependencies

New metadata field

  • Debian — system-level dependencies

Hints for system-level dependencies

Presenter Notes

PEP 345 - Metadata v1.2

  • project-level dependencies: Requires-Dist
  • Environment markers: OS-specific fields in pseudo-Python
  • hints for system-level dependencies: Requires-External
  • implemented in Packaging

PKG-INFO examples:

Requires-Dist: lxml (> 1.0)
Requires-Dist: zope.interface (> 1.0)
Requires-Dist: pywin32 (>1.0);
               sys.platform == 'win32'
Requires-External: libxslt-dev

Presenter Notes

パッケージングで一般的な5つの課題

  1. How do I define dependencies ? / 依存関係をどう定義するか?
  2. what's setup.py role ? / setup.py の役割は何か?
  3. what's installed ? / 何がインストールされるか?
  4. what about versions schemes ? / バージョン管理の仕組みは?
  5. how to define data files ? / データファイルをどう定義するか?

Presenter Notes

setup.py ?

Presenter Notes

setup.py roles / 役割

  • register and upload a project at PyPI
  • build the project
  • develop with it
  • get information about the project (metadata, files..)
  • install the project
  • ...

Presenter Notes

"Le" setup.py

Setup.py

Presenter Notes

setup.cfg

  • static representation for:
    • metadata
    • project content
  • non-intrusive for older tools
  • clean separation from build process

Presenter Notes

Setup.cfg example / 例

 1 [metadata]
 2 name = gitbuster
 3 version = 0.9
 4 author = Julien Miotte
 5 author_email = julien@gmail.com
 6 summary = Python Qt4 frontend
 7 description_file = README.rst
 8 home_page = https://gitbuster
 9 requires_dist = gfbi_core (>=0.1)
10 requires_python = >=2.5

Presenter Notes

Setup.cfg example / 例

 1 [files]
 2 packages =
 3     gitbuster
 4 scripts =
 5     gitbuster/gitbuster
 6 extra_files =
 7     setup.cfg
 8     setup.py
 9     README.rst
10     AUTHORS.txt
11     demo.sh
12     LICENSE.txt
13     MANIFEST.in

Presenter Notes

Wizard Demo ! / ウィザードデモ!

Wizard demo

Presenter Notes

パッケージングで一般的な5つの課題

  1. How do I define dependencies ? / 依存関係をどう定義するか?
  2. what's setup.py role ? / setup.py の役割は何か?
  3. what's installed ? / 何がインストールされるか?
  4. what about versions schemes ? / バージョンをどう付けるか?
  5. how to define data files ? / データファイルをどう定義するか?

Presenter Notes

何がインストールされるか?

Presenter Notes

What's installed ?

How to:

  • uninstall ?!?
  • list what's installed

Different Installation formats:

  • Distutils — nothing recorded (but --record exists!)
  • easy_install — installation not recorded
  • Pip — records in install-record.txt

Presenter Notes

PEP 376 - Database of installed projects

  • Standard Database of installed projects
  • Packaging implements it

Layout:

  • site-packages
    • Project.dist-info
      • RECORD
      • METADATA
      • INSTALLER
      • REQUESTED

Presenter Notes

PEP 376 features / 機能

  • Iterate on installed projects:
  • List installed files
  • Know which project uses which file
  • Uninstall !

Presenter Notes

Presenter Notes

パッケージングで一般的な5つの課題

  1. How do I define dependencies ? / 依存関係をどう定義するか?
  2. what's setup.py role ? / setup.py の役割は何か?
  3. what's installed ? / 何がインストールされるか?
  4. what about versions schemes ? / バージョンをどう付けるか?
  5. how to define data files ? / データファイルをどう定義するか?

Presenter Notes

バージョンの付け方

Presenter Notes

実際に PyPI でリリースされたバージョン

  • working proof of concept
  • dev
  • TRUNKdev-r5
  • unreleased.unofficialdev
  • 1.0dev-BZR-r45-panta-elasticworld.org-20091021145839-1oceeh3stpvyl04t
  • 1.01b1+encoding_patch+removed_django_depends
  • 0.19 "Nose Furnace"
  • (tip)
  • Pre-Alpha

Presenter Notes

PEP 386 -- Standard version scheme

  • MAJOR.MINOR[.MICRO]...
  • Included in Metadata 1.2
  • PyPI implements it

Examples:

1.1 < 1.2a1 < 1.2 < 1.3.dev2 < 1.3 < 1.3.2

Presenter Notes

パッケージングで一般的な5つの課題

  1. How do I define dependencies ? / 依存関係をどう定義するか?
  2. what's setup.py role ? / setup.py の役割は何か?
  3. what's installed ? / 何がインストールされるか?
  4. what about versions schemes ? / バージョンをどう付けるか?
  5. how to define data files ? / データファイルをどう定義するか?

Presenter Notes

データファイルの定義

Presenter Notes

Distutils & とデータファイル

  • Python modules
  • Scripts
  • "Data" files -- explicit paths

Example of the __file__ hack:

1 import os.path
2 here = os.path.dirname(__file__)
3 
4 myfile = open(os.path.join(here, 'pic.jpg'))

Presenter Notes

データファイル定義の新しい方法

One level of indirection:

  1. define resources in setup.cfg
  2. installation time: sysconfig.cfg lookup
  3. setup.cfg definitions copied in RESOURCES (metadata)
  4. runtime: use get_file

Step #1 — You define in setup.cfg:

1 [files]
2 packages = foo
3 resources =
4   pic.jpg = {appdata}
5   app/app.icon = {icon}
6   somestuff.txt = {datadir}/{dist.name}

Presenter Notes

データファイル定義の新しい方法

Step #2 — The installer look at sysconfig.cfg (extract):

1 [posix]
2 datadir = /usr/share
3 appdata = {datadir}/{dist.name}

Example: where to install pic.jpg ?

  • project name : MyProject
  • platform: posix
  • setup.cfg definition: pic.jpg = {appdata}

pic.jpg installed in /usr/share/MyProject/

Presenter Notes

データファイル定義の新しい方法

Step #3 — You get your file back:

1 from packaging.database import get_file
2 
3 icon = get_file('MyProject', 'app/app.icon')
4 pic = get_file('MyProject', 'pic.jpg')

get_file process:

  1. locate MyProject, open its RESOURCES file
  2. find pic.jpg = {appdata}
  3. read sysconfig.cfg
  4. Find the file

Presenter Notes

Packaging Roadmap / ロードマップ

  • backport in 2.x : "distutils2" -- ETA end of summer
  • version 1.0 in Python 3.3
  • support setuptools/distribute and packaging in your projects

Presenter Notes

いま風の 5 Tips

Presenter Notes

Tip #1 — PEP 386 準拠のバージョン体系を使う

Presenter Notes

Tip #2 — できるだけひとまとめに setup.py を作る

Presenter Notes

Tip #3 — インストーラーの利用を前提としない

Presenter Notes

Tip #4 — PyPI に不安定版をリリースしない

Presenter Notes

Tip #5 — データファイルに気を付ける

Presenter Notes

Python 3

Presenter Notes

Python 3

  • py3k branch started in Q1 2006
  • 3.0 released in November 2008
  • 3.1 released in June 2009
  • 3.2 released in February 2011

Presenter Notes

Slow but constant uptake

py3pkgs

Presenter Notes

Available under Python 3

  • Distribute
  • Numpy/Scipy
  • SQLAlchemy
  • CherryPy
  • Bottle

See: https://py3ksupport.appspot.com/

Presenter Notes

Missing libs & frameworks

  • PyCrypto
  • WebOb (very soon)
  • NLTK
  • Twisted
  • Django
  • Pyramid (~8 months)

Presenter Notes

PyPy

  • Faster than CPython in some cases !
  • Syntax: Python 2.7
  • Django, ctypes, etc..
  • Features: sandboxing, stackless, etc
  • No support for gevent, some C extensions

Presenter Notes

Python 2.x v.s. Python 3 (vs PyPy)

Presenter Notes

どうも !

Questions ?

Twitter: @tarek_ziade Mail: tarek@ziade.org

Presenter Notes

Presenter Notes