Happy Dussehra! May good always win. Happy Diwali! Wishing you a bright year ahead. Merry Christmas! Warm wishes from Vizag. Happy New Year! Here’s to a great year ahead.

Archive ยท Linux

Python 2 Reaches End of Life

What the end of Python 2 means for servers and scripts, how to find every use of it, and a safe migration plan with testing and fallbacks.

3 min read

  • Ubuntu 18.04.3 LTS
  • Python 3.6
  • CentOS 8.1
  • Python 2.7

Python 2.7 was officially retired on 1 January 2020. The core team no longer fixes bugs or security problems in it. One last release, 2.7.18, is planned for April 2020 to tidy up, and after that Python 2 receives nothing from upstream. Distributions may patch their own packages for a while, but the interpreter and most libraries are moving on.

Versions in use on this date. Ubuntu 18.04.3 LTS ships Python 3.6 and keeps Python 2.7 available for compatibility. CentOS 7.7 still depends on Python 2.7 for yum, so do not remove it there. CentOS 8.1 (released November 2019) is Python 3 first and has no default python command at all.

Find where Python 2 is used

flowchart TB
  S[Inventory servers and repositories] --> A[Scripts with python shebang]
  S --> B[Cron jobs and systemd units calling python]
  S --> C[Config management and deploy tools]
  S --> D[Third-party packages in requirements files]
  A --> T[Test under Python 3]
  B --> T
  C --> T
  D --> T
  T --> R{Passes?}
  R -->|yes| M[Change shebang and deploy]
  R -->|no| F[Fix code or pin a Python 3 compatible package]
  F --> T
python --version 2>&1              # may be missing, or 2.7
python3 --version                  # 3.6.x on Ubuntu 18.04
sudo grep -rIl --include='*' -E '^#!.*python($|[^0-9])' /opt /usr/local/bin /home 2>/dev/null
sudo grep -rn "python " /etc/cron* /etc/systemd/system 2>/dev/null
pip list 2>/dev/null | head        # packages installed for Python 2
pip3 list | head

Also check anything that shells out to python: deployment scripts, Ansible tasks (ansible_python_interpreter), monitoring plugins, and old Nagios checks.

What changes between 2 and 3

Area Python 2 Python 3
print statement: print "x" function: print("x")
Division 5/2 == 2 5/2 == 2.5, use // for integers
Strings str is bytes, unicode is text str is text, bytes is separate
Dictionaries .iteritems(), .has_key() .items(), in
Exceptions except Exception, e except Exception as e
Input raw_input() input()
Imports urllib2, ConfigParser urllib.request, configparser

The text-versus-bytes change causes most real failures, especially when reading files, sockets or subprocess output. Be explicit: open(f, encoding="utf-8") and .decode("utf-8").

Migration steps

  1. Pin and test first. Record dependencies with pip freeze > requirements.txt and make sure there are at least basic tests; a script without tests needs a manual run on sample data.
  2. Automatic conversion. 2to3 -w script.py (included with Python 3) rewrites most syntax, writing a diff you must read. python-modernize or futurize produce code that works under both versions, useful for a staged migration.
  3. Run under Python 3 in staging, with the same inputs as production, and compare output files byte for byte.
  4. Check libraries. Most popular libraries support Python 3; where one does not, find the maintained replacement. Pin versions that support 3.6.
  5. Change the shebang to #!/usr/bin/env python3 and the cron or systemd command to python3. Use a virtualenv per application: python3 -m venv /opt/app/venv && /opt/app/venv/bin/pip install -r requirements.txt.
  6. Deploy gradually, keeping the old script available for rollback for a few weeks.

Server-level notes

  • Ubuntu 18.04: python3 is installed, python may not exist. Do not point python at Python 3 globally with update-alternatives; fix scripts instead, because other packages may assume Python 2 behaviour.
  • CentOS 7: yum and several tools need /usr/bin/python to remain 2.7. Leave it alone; install Python 3 for your applications (from EPEL or the Software Collections).
  • CentOS 8: choose explicitly: sudo alternatives --set python /usr/bin/python3 or call python3 everywhere.
  • Ansible: set ansible_python_interpreter=/usr/bin/python3 for hosts that support it.

Risk and what to do about leftovers

A Python 2 script that processes untrusted input is now a growing security risk, since new flaws will not be fixed upstream. If you must keep one running, isolate it in a container or restricted account, limit its network access, and put a replacement on the plan with a date. Record every remaining Python 2 dependency in your inventory so it does not become a surprise at the next audit.