Namespaces, referrers, referents, inheritance and confusion

Struggling with ways of saving echonest audio objects for later use has lead down the road of multiprocessing (probably misguidedly), into pickling and from there to the dill package (library), which opened a door to some ignorance that may be leading into further understanding of namespaces, referrers, referents and inheritance.

See, Dill includes two modules which detect “parents” and “children”. Hmmm…

dill.detect.children

children(obj, objtype, depth=1, ignore=())
"""Find the chain of referrers for obj. Chain will start with obj.

objtype: an object type or tuple of types to search for
depth: search depth (e.g. depth=2 is 'grandchildren')
ignore: an object or tuple of objects to ignore in the search

NOTE: a common thing to ignore is all globals, 'ignore=(globals(),)'

NOTE: repeated calls may yield different results, as python stores
the last value in the special variable '_'; thus, it is often good
to execute something to replace '_' (e.g. >>> 1+1).

"""

Remember here that a reference is a value that enables a access to some data.

The term referent means the object which is referred to, and the referers are the objects that refer to other objects. Being that the “chain of referrers” are called “children” (and grandchildren) we would intuitively expect that they would not exist without their parents (or grandparents), but that’s not the case.

Spent a good bit of time trying to fully comprehend what is meant by “parents” and “children” in terms of reference, as opposed to inheritance, and ended up also installing an incredible module (or package – let’s use the term, Library) called objGraph, upon which some of Dill is based.

Consider the following:

 
#!/usr/bin/env python
# encoding: utf=8

"""
python pydill.py

"""
import dill
import types
import objgraph

            
def adder(x, y):
    return x + y

class GrandParent(object):

    lastname="Smith"

    def gp_adder(self, y):
        return adder(self.x, self.y)
        
class Parent(GrandParent):
    pass
    
class Child(Parent):
    name="Fredly"

only_child = Child()

print ("Children:")
# points forward from "grand_child" in "globals" in "__main__"
print(dill.detect.children(only_child, types.ModuleType, depth=2, ignore=(globals(),)))
objgraph.show_backrefs(only_child, max_depth=3, filename='only_child_children'+'.png')

print ("Parents:")
# points backward from Child from Parent from Grandparent
print(dill.detect.parents(only_child, dict, depth=2, ignore=(globals(),)))
objgraph.show_refs(only_child, max_depth=4, filename='only_child_parents'+'.png')


 

(Note use of the Types module.)

Child returns:

 [< module '__main__' from 'pydill.py'>, {'dill': < module 'dill' from '/Users/mikekilmer/Envs/GLITCH/lib/python2.7/site-packages/dill/__init__.pyc'>, 'objgraph': < module 'objgraph' from '/Users/mikekilmer/Envs/GLITCH/lib/python2.7/site-packages/objgraph.pyc'>, 'Parent': < class '__main__.Parent'>, '__builtins__': < module '__builtin__' (built-in)>, '__file__': 'pydill.py', 'Child': < class '__main__.Child'>, '__package__': None, 'adder': < function adder at 0x10251ce60>, 'GrandParent': , '__name__': '__main__', 'only_child': <__main__.Child object at 0x1024fed10>, '__doc__': '\npython pydill.py\n\n', 'types': < module 'types' from '/Users/mikekilmer/Envs/GLITCH/lib/python2.7/types.pyc'>}, <__main__.Child object at 0x1024fed10>]
 

Which is a list of three items; essentially: [__main__, globals(), grand_child]. The first is the `__main__` module from the file itself, the third is Parent object itself, which is the only object we would find if we had specified `depth=1`, and the middle item is a dictionary object, one of which is created anytime python is executing, and the contents of which are returned by the function ‘globals()’.

The default content of what is commonly referred to as ‘globals()‘ is:

{'__builtins__': , '__name__': '__main__', '__doc__': None}

In this case the ‘globals()’ dictionary, returned at depth=2 (At depth=1 detect.children would only return the Child() object.) on our Parent() instance contains thirteen items who’s keys are:

'dill', 'objgraph', 'Parent', '__builtins__', '__file__', 'Child', 'only_child', '__package__', 'adder', 'GrandParent', '__name__', '__doc__', 'types'

So there are three objects that are children, or referrers of (refer to) our our Parent() instance (aka only_child), of which one ‘globals()’, contains multiple other referents (Parents), to which it refers.

To understand ‘globals()’, let’s look at the concept of namespace.

A namespace is a container in which a certain name can have a certain meaning. For example within the namespace of this article, the pronoun I refers to me. Two links, or even one link away, the pronoun I might refer to someone completely different. If I was to quote someone else, like for example, MLK saying, “I have a dream.” There was another namespace created within those quotes in which the pronoun I refers to MLK.

As with many programming languages, any Python function, method or loop creates it’s own namespace so that within three different functions, i=1, i=x and i=”something completely” won’t interfere with each other. Each namespace
in python can be described by a dictionary of string names of
the objects, and the corresponding objects themselves.

Well, each time Python runs a script it creates an outermost namespace with an object called ‘__main__’, and the dictionary that is part of ‘__main__’ can be called with the function, globals(). And this dictionary includes all of the objects that are available within the outermost namespace of the script, which includes Classes, functions, imported modules, etc. Above we see the name of the object, ‘only_child’, for example, and the corresponding object <__main__.Parent object …>. The dictionary returned by globals() refers to ‘only_child’, as well as any other object that is accessible at the outermost namespace within a given Python execution environment.

Check out the graphic rendition of the “children” of ‘only_child’ as rendered by objGraph:

“Children” is a confusing way to refer to referrers because ‘__main__’ didn’t give birth to ‘only_child’, but ‘__main__’ refers to ‘only_child’ in the same way that a child refers to it’s parents in the way the child looks, speaks, etc. So the children are the referrers (aka back_refs), and the parents are the referred to, otherwise known as the referents.

In this case dill.detect.parents() returns:

[<__main__.Child object at 0x1024fed10>, , {'__module__': '__main__', '__doc__': None, 'firstname': 'Fredly'}]

In python, “=” is actually assignment by pointer
reference…. so you read “only_child = Child()” as “only_child
points to an instance of the Child class”, and here we see the Child object, the Child class and the dictionary object contained within the Child class.

Note that if we make any changes to the instance.attributes, dill will return the instances dictionary. For example:

married_child = Child()
married_child.lastname = "Borrower"
print(dill.detect.parents(married_child, dict, depth=2))
objgraph.show_refs(married_child, max_depth=3, filename='married_child_parents'+'.png')

returns:

[<__main__.Child object at 0x1024fdd10>, {'lastname': 'Borrower'}]

Check out the objGraph images:

Further (and gentle) exposure of my ignorance is welcome, and so are you.