Python Programming Language

My Python Stuff

• My PyCon 2010 notes
• See posts and notebooks tagged python
  • My Happy Python Workflow
  • Getting Out of the Programming Goldilocks Zone

Meta

• Awesome Python

Advanced Python

dabeaz-course/python-mastery: Advanced Python Mastery (course by David Beazley)

Books

• Fundamentals of Python programming (PDF) — ”… introduction to computer programming textbooks are available in PDF form for free download. University and high school students and self learners around the world have found them helpful. See the terms of use in the preface of each book.”

• Intermediate Python — “Fluent python” by Luciano Ramalho.

Learning sources

• Tiny-Python-3.6-Notebook/python.rst at master · mattharrison/Tiny-Python-3.6-Notebook

“This repository contains the text for the Tiny Python 3.6 Notebook. it is a notebook containing curated examples for Python 3 as well as the new features found in Python 3.6.”

Internals

• How virtual environments work
• CPython Dynamic Dispatch Internals: How a + b Works?

Developer Experience (DevEx)

Make Python DevEx — this is very close to how I choose to do Python development. good read. I don’t have such an extensive Makefile setup.

Debugging

• Debugging cpython with gdb

• Martin Heinz - “Ultimate Guide to Python Debugging” 2020-06-02. Talks a lot about logging, but there is some interesting bits about using __repr__ and __missing__ dunder methods.

Profiling

• benfred/py-spy: Sampling profiler for Python programs

• Scalene: a high-performance, high-precision CPU and memory profiler for Python has interleaved cross-language profiling.

• Pyflame: A Ptracing Profiler For Python — Pyflame 1.4.0 documentation

Web

• emmett-framework/granian: A Rust HTTP server for Python applications
• Robyn - An async Python Framework written in Rust.

Multiprocessing

• Stabbing yourself with a fork() in a multiprocessing.Pool full of sharks

Async

• Concurrency and async / await - FastAPI

Sublime Text

• Flake8Lint is infuriating but important plugin for sublime text. Flake8 Rules

Alternate Python implementations

• Zippy ; Slides, paper

To Install Zippy:

hg clone https://bitbucket.org/ssllab/zippy
cd zippy
TBD

Libraries

• Deciphering Glyph :: Python Packaging Is Good Now; August 2016.
• Cookiecutter — A command-line utility that creates projects from cookie cutters (project templates). See Example using cookie cutter for pandoc presentations.
• Python modules you should know
• 20 Python Libraries You Aren’t Using (But Should) [PDF] by Caleb Hattingh on O’Reilly. (broken link; sep 2024)
• Peep. A “pip install” that is cryptographically guaranteed repeatable
• imutils — A series of convenience functions to make basic image processing operations such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV and Python.
• The Python Graph Gallery – Visualizing data – with Python
• Getting the Most out of Sqlite3 with Python; See dataset and apsw.
• mahmoud/awesome-python-applications: 💿 Free software that works great, which also happens to be open-source Python.

Data science libraries

• agate 1.6.0 — agate 1.6.0 documentation
• dataset: databases for lazy people
• Python Itertools: For a faster and memory efficient code – Kanoki

Conferences

• PyData (NYC 2013).
• Videos.

Python3

• The key differences between Python 2.7.x and Python 3.x with examples
• “the not-so-simple intricacies of Python2→Python3” - oilshell comments on Python 3 and Firefox 57 (an observation)

Pypy

• PyPy Status Blog: The First 15 Years of PyPy — a Personal Retrospective

Package management

• Python packages with pyproject.toml and nothing else | Simon Willison’s TILs; Jul 2023. Discovered it via My User Experience Porting Off setup.py

• Python Project Tooling explained — chobeat; Apr 2019. lobste.rs

  • Poetry
  • Pipsi
  • pyenv (polyglot alternate is mise; is a replacement for pyenv, rbenv, asdf, nvm, etc.,)
  • not mentioned above, but see also hatch and pdm

• Use pew, not virtualenvwrapper, for Python virtualenvs

• Use twine, and wheel.

• Follow the packaging tutorial

• Use TestPyPi

• Short summary of commands:

python setup.py sdist
python setup.py bdist_wheel
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

• uv: An Extremely Fast Python Package Manager - YouTube, talk by Charlie Marsh of Astral.sh

Project Managers

Do not use requirements.txt - quanttype — instead choose Poetry, Hatch or pdm.

Poetry

I have been using poetry for a while.

Poetry uses the pyproject.toml standard, but it does not follow the standard specifying how metadata should be represented in a pyproject.toml file (PEP 621), instead using a custom [tool.poetry] table. This is partly because Poetry came out before PEP 621. — from pdm’s documentation

Hatch

• Standardized build system with reproducible builds by default
• Robust environment management with support for custom scripts
• Easy publishing to PyPI or other indexes
• Version management
• Configurable project generation with sane defaults
• Responsive CLI, ~2-3x faster than equivalent tools

pdm

Unlike Poetry and Hatch, PDM is not limited to a specific build backend; users have the freedom to choose any build backend they prefer.

Links

• Solving the Water Jug Problem from Die Hard 3 with TLA+ and Hypothesis
• Writing beautiful intelligent python — talk by Raymond Hettinger in Pycon 2015.
• profiling - Python profiler. HN; Aug 2015
• vprof - Visual Profiler; flame charts, memory usage etc.,
• Pyparallel — “An experimental, proof-of-concept fork of Python 3 designed to optimally exploit multiple CPU cores, fast SSDs, NUMA architectures and 10Gb+ Ethernet networks. Removes the limitation of the Global Interpreter Lock (GIL) without needing to remove it at all.”
• Replacing Python: Candidates. A survey of various languages while keeping these considerations in mind: speed, binary compatibility, safety, diagnostics, ease of writing, shared libs, static typing, bounds of privilege, mutability, C interoperability and asynchronous code. OCaml appears to be the winner. This is written by the author of 0install.
• Setting up Emacs for python development including Pymacs, pyflakes, pep8, python-mode.el, python.el etc.,. Pymacs has detailed documentation (31pp) on their website.
• Python Tips — Intermediate Python
• Nov 2015; Python for High performance scintific computing (disussion)
• Python: Generators, Coroutines, Native Coroutines and async/await | Abu Ashraf Masnun -Parallelizing a Python Function for the Extremely Lazy - tdhopper.com
• The Little Book of Python Anti-Patterns — Python Anti-Patterns documentation

2019

Via My Python Development Environment, 2020 Edition | Jacob Kaplan-Moss:

• pipx · PyPI
• Poetry - Python dependency management and packaging made easy.
• VisiData