I tried to graph the packages installed in Python

result

-> PyPI Top 100 Package Version: http://qiita.com/kitsuyui/items/a7cc56d476b59ff07f54

I got this figure with my virtualenv for machine learning. It's obvious at a glance.

I thought numpy and scipy were just as dependent, scipy is less dependent ...

The maintainer's struggle is exuded by the fact that many packages use six to make Python 3 and Python 2 compatible.

In the rest of the sections, pip freeze and pipdeptree, the script that outputs this figure is itemized.

pip freeze

python

$ pip freeze -l

Then you can see the packages in the current virtualenv. However, since there is no hierarchy and everything is described flat, it comes out in a row and I do not understand the dependency.

autoflake==0.6.6
autopep8==1.1.1
flake8==2.2.4
graphviz==0.4.3
hacking==0.10.1
isort==3.9.6
matplotlib==1.4.3
mccabe==0.2.1
natsort==3.5.6
nose==1.3.6
numpy==1.9.2
pandas==0.16.0
pbr==0.10.8
pep8==1.5.7
pies==2.6.3
pipdeptree==0.4.2
py==1.4.26
pyflakes==0.8.1
pyparsing==2.0.3
pytest==2.7.0
python-dateutil==2.4.2
pytz==2015.2
scipy==0.15.1
six==1.9.0
Theano==0.7.0
tox==1.9.2
virtualenv==12.1.1

Use pipdeptree

On the other hand, pipdeptree can be used to display the dependency hierarchy in an easy-to-understand manner.

python

$ pipdeptree --nowarn --local-only --freeze --all

autoflake==0.6.6
  - pyflakes==0.8.1
autopep8==1.1.1
  - pep8==1.5.7
flake8==2.2.4
  - pyflakes==0.8.1
  - pep8==1.5.7
  - mccabe==0.2.1
graphviz==0.4.3
hacking==0.10.1
  - pyflakes==0.8.1
  - six==1.9.0
  - flake8==2.2.4
    - pyflakes==0.8.1
    - pep8==1.5.7
    - mccabe==0.2.1
  - pep8==1.5.7
  - mccabe==0.2.1
  - pbr==0.10.8
    - pip
isort==3.9.6
  - pies==2.6.3
  - natsort==3.5.6
matplotlib==1.4.3
  - nose==1.3.6
  - python-dateutil==2.4.2
    - six==1.9.0
  - pyparsing==2.0.3
  - six==1.9.0
  - pytz==2015.2
  - numpy==1.9.2
mccabe==0.2.1
natsort==3.5.6
nose==1.3.6
numpy==1.9.2
pandas==0.16.0
  - pytz==2015.2
  - python-dateutil==2.4.2
    - six==1.9.0
  - numpy==1.9.2
pbr==0.10.8
  - pip
pep8==1.5.7
pies==2.6.3
py==1.4.26
pyflakes==0.8.1
pyparsing==2.0.3
pytest==2.7.0
  - py==1.4.26
python-dateutil==2.4.2
  - six==1.9.0
pytz==2015.2
scipy==0.15.1
six==1.9.0
Theano==0.7.0
  - numpy==1.9.2
  - scipy==0.15.1
tox==1.9.2
  - virtualenv==12.1.1
  - py==1.4.26
virtualenv==12.1.1

Let's make a graph.

I decided to use graphviz. The following is a script that passes the output of pipdeptree as standard input and graphs it with graphviz. Like my other articles, I use Python 3.

graphout.py

# coding: utf-8
import re

import graphviz


def depfile_to_tree(file_like_object):
    line_pattern = re.compile(r'^(?:(?P<indent>(?:  )*)- )?'
                              r'(?P<pkg_name>.*?)'
                              r'(?:==(?P<version>.*?))?\n$')

    tree = []

    for line in file_like_object:
        matcher = line_pattern.match(line)
        indent, pkg_name, version = matcher.group('indent',
                                                  'pkg_name',
                                                  'version')

        if indent is None:
            hierarchy = 0
        else:
            hierarchy = int(len(indent) / 2)

        if hierarchy < len(tree):
            tree = tree[:hierarchy]

        tree.append((pkg_name, version))
        yield tuple(tree)


def main(pipdeptree_file):
    packages = set()
    dependencies = set()

    for package_tree in depfile_to_tree(pipdeptree_file):
        pkg_name = package_tree[-1][0]
        packages.add(pkg_name)
        if len(package_tree) < 2:
            continue
        parent, child = package_tree[-2:]
        dependencies.add((parent[0], child[0], child[1]))

    dot = graphviz.Digraph(comment='My Pip Dependencies')
    for p in packages:
        dot.node(p)

    for d in dependencies:
        dot.edge(*d)

    dot.engine = 'dot'
    dot.format = 'png'
    dot.render('dependencies.gv')

if __name__ == '__main__':
    import sys
    main(sys.stdin)

How to use

python

$ pipdeptree --nowarn --local-only --freeze --all > dependencies.txt
$ python graphout.py < dependencies.txt

Generate dependencies.gv and dependencies.gv.png.

Digression

It seems that pip does not resolve true dependencies. It may be solved by using pip-tools (pip-review) or depsolver. Absent.