In the publication I briefly describe what metaclasses are in Python. The writeup is a living item and will be extended with more detail shortly.
Please enjoy and comment.
1. Classes as objects
One liner: Metaclasses = 'stuff' that creates class-objects = a 'class factory'. You can
create metaclass 2 ways: (1) derive from “type” class or (2) set __metaclass__
keyword's value inside class body to a class/type.
Class describe objects:
>>> class ObjectCreator(object): pass
>>> my_object = ObjectCreator()
>>> print(my_object)
<__main__.ObjectCreator object at 0x8974f2c>
But in Python classes are objects too.
Python creates a class “object” with this:
>>> class ObjectCreator(object): pass
This object is itself capable of creating objects (the instances).
Since class is object, you can – assign it to variable, copy, add attributes, pass into functions.
e.g.:
>>> print(ObjectCreator) # print a class!
<class '__main__.ObjectCreator'>
>>> def echo(o):
... print(o)
...
>>> echo(ObjectCreator) # pass a class as parameter!
<class '__main__.ObjectCreator'>
>>> print(hasattr(ObjectCreator, 'new_attribute'))
False
>>> ObjectCreator.new_attribute = 'foo' # add attributes to class!
>>> print(hasattr(ObjectCreator, 'new_attribute'))
True
>>> print(ObjectCreator.new_attribute)
foo
>>> ObjectCreatorMirror = ObjectCreator # assign class to variable!
>>> print(ObjectCreatorMirror.new_attribute)
foo
>>> print(ObjectCreatorMirror())
<__main__.ObjectCreator object at 0x8997b4c>
>>> def choose_class(name):
... if name == 'foo':
... class Foo(object): # create class inside function!
... pass
... return Foo # return a class like object!
... else:
... class Bar(object):
... pass
... return Bar
2. type can create classes on the fly. type can take description and return a class.
type(name of class, tuple of parent classes, dict of attributes)
>>> class MyShinyClass(object): pass
IS SAME AS ...
>>> MyShinyClass = type('MyShinyClass', (), {}) # returns a class object
>>> print(MyShinyClass)
<class '__main__.MyShinyClass'>
>>> print(MyShinyClass())
<__main__.MyShinyClass object at 0x8997cec>
So, this is same as that –
>>> class Foo(object):
... bar = True
>>> Foo = type('Foo', (), {'bar':True}) #add class attributes
>>> print(Foo)
<class '__main__.Foo'>
>>> print(Foo.bar)
True
inherit from it...
>>> class FooChild(Foo): pass
or...
>>> FooChild = type('FooChild', (Foo,), {}) #inherit from it
>>> print(FooChild)
<class '__main__.FooChild'>
To add class-methods ...
>>> def echo_bar(self):
... print(self.bar)
...
>>> FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar}) #add methods
>>> hasattr(Foo, 'echo_bar')
False
>>> hasattr(FooChild, 'echo_bar')
True
>>> my_foo = FooChild()
>>> my_foo.echo_bar()
True
type is the metaclass Python uses to create all classes behind the scenes.
MyClass = MetaClass()
MyObject = MyClass()
SAME AS … MyClass = type('MyClass', (), {})
You see that by checking the __class__ attribute.
3. >>> age = 35
>>> age.__class__
<type 'int'>
>>> name = 'bob'
>>> name.__class__
<type 'str'>
>>> def foo(): pass
>>> foo.__class__
<type 'function'>
>>> class Bar(object): pass
>>> b = Bar()
>>> b.__class__
<class '__main__.Bar'>
__class__ of any __class__ is … type
>>> age.__class__.__class__
<type 'type'>
>>> name.__class__.__class__
<type 'type'>
>>> foo.__class__.__class__
<type 'type'>
>>> b.__class__.__class__
<type 'type'>
type is the built-in metaclass Python uses, but of course, you can create your own metaclass.
The __metaclass__ attribute
You can add a __metaclass__ attribute when you write a class:
class Foo(object):
__metaclass__ = something that can create a class = type or any inherited from it
…
When you do: class Foo(Bar): pass
Python does the following:
Is there a __metaclass__ attribute in Foo? If yes, create in memory a class object named Foo
by using what is in __metaclass__. If Python can't find __metaclass__, it will look for a
__metaclass__ at the MODULE level, and try to do the same.
Then if it can't find any __metaclass__ at all, it will use the Bar's (the first parent) own
metaclass (which might be the default type) to create the class object.
Be careful here that the __metaclass__ attribute will not be inherited, the metaclass of the parent
(Bar.__class__) will be. If Bar used a __metaclass__ attribute that created Bar with type() (and not
type.__new__()), the subclasses will not inherit that behavior.
In __metaclass__ field, you can put something like type or anything that subclasses or
uses it.
4. Custom metaclasses
The main purpose of a metaclass is to change the class automatically, when it's created.
Imagine you decide that all classes in your module should have their attributes written in
uppercase. There are several ways to do this, but one way is to set __metaclass__ at the
module level.
This way, all classes of this module will be created using this metaclass, and we just have to
tell the metaclass to turn all attributes to uppercase.
Luckily, __metaclass__ can actually be any callable, it doesn't need to be a formal class.
# the metaclass will automatically get passed the same argument that you usually pass to `type`
def upper_attr(future_class_name, future_class_parents, future_class_attr):
"""
Return a class object, with the list of its attribute turned
into uppercase.
"""
# pick up any attribute that doesn't start with '__' and uppercase it
uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val
# let `type` do the class creation
return type(future_class_name, future_class_parents, uppercase_attr)
__metaclass__ = upper_attr # this will affect all classes in the module
class Foo(): # global __metaclass__ won't work with "object" though
# but we can define __metaclass__ here instead to affect only this class
# and this will work with "object" children
self.bar = 'bip'
print(hasattr(Foo, 'bar'))
# Out: False
print(hasattr(Foo, 'BAR'))
# Out: True
f = Foo()
print(f.BAR)
# Out: 'bip'
Now, let's do exactly the same, but using a real class for a metaclass:
# remember that `type` is actually a class like `str` and `int`
# so you can inherit from it
class UpperAttrMetaclass(type):
# __new__ is a “staticmethod”, called before __init__
# it's the method that creates the object and returns it
# while __init__ just initializes the object passed as parameter
# you rarely use __new__, except when you want to control how the object
5. # is created.
# here the created object is the class, and we want to customize it
# so we override __new__
# you can do some stuff in __init__ too if you wish
# some advanced use involves overriding __call__ as well, but we won't
# see this
def __new__(upperattr_metaclass, future_class_name,
future_class_parents, future_class_attr):
uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val
return type(future_class_name, future_class_parents, uppercase_attr)
But this is not really OOP. We call type directly and we don't override call the parent
__new__. Let's do it:
class UpperAttrMetaclass(type):
def __new__(upperattr_metaclass, future_class_name,
future_class_parents, future_class_attr):
uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val
# reuse the type.__new__ method
# this is basic OOP, nothing magic in there
return type.__new__(upperattr_metaclass, future_class_name,
future_class_parents, uppercase_attr)
You may have noticed the extra argument upperattr_metaclass. There is nothing special
about it: a method always receives the current instance as first parameter. Just like you have
self for ordinary methods.
Of course, the names I used here are long for the sake of clarity, but like for self, all the
arguments have conventional names. So a real production metaclass would look like this:
class UpperAttrMetaclass(type):
def __new__(cls, clsname, bases, dct):
uppercase_attr = {}
for name, val in dct.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val
return type.__new__(cls, clsname, bases, uppercase_attr)
6. We can make it even cleaner by using super, which will ease inheritance (because yes, you
can have metaclasses, inheriting from metaclasses, inheriting from type):
class UpperAttrMetaclass(type):
def __new__(cls, clsname, bases, dct):
uppercase_attr = {}
for name, val in dct.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val
return super(UpperAttrMetaclass, cls).__new__(cls, clsname, bases, uppercase_attr)
That's it. There is really nothing more about metaclasses.
The reason behind the complexity of the code using metaclasses is not because of
metaclasses, it's because you usually use metaclasses to do twisted stuff relying on
introspection, manipulating inheritance, vars such as __dict__, etc.
Indeed, metaclasses are especially useful to do black magic, and therefore complicated stuff.
But by themselves, they are simple:
intercept a class creation
modify the class
return the modified class
Why would you use metaclasses classes
instead of functions?
Since __metaclass__ can accept any callable, why would you use a class since it's obviously
more complicated?
There are several reasons to do so:
The intention is clear. When you read UpperAttrMetaclass(type), you know what's
going to follow
You can use OOP. Metaclass can inherit from metaclass, override parent methods.
Metaclasses can even use metaclasses.
You can structure your code better. You never use metaclasses for something as trivial
as the above example. It's usually for something complicated. Having the ability to
make several methods and group them in one class is very useful to make the code
easier to read.
You can hook on __new__, __init__ and __call__. Which will allow you to do different
stuff. Even if usually you can do it all in __new__, some people are just more
7. comfortable using __init__.
These are called metaclasses, damn it! It must mean something!
Use of metaclasses
Python Guru Tim Peters
The main use case for a metaclass is creating an API. A typical example of this is the Django
ORM.
It allows you to define something like this:
class Person(models.Model):
name = models.CharField(max_length=30)
age = models.IntegerField()
But if you do this:
guy = Person(name='bob', age='35')
print(guy.age)
It won't return an IntegerField object. It will return an int, and can even take it directly from the
database.
This is possible because models.Model defines __metaclass__ and it uses some magic that
will turn the Person you just defined with simple statements into a complex hook to a
database field.
Django makes something complex look simple by exposing a simple API and using
metaclasses, recreating code from this API to do the real job behind the scenes.
The last word
First, you know that classes are objects that can create instances.
Well in fact, classes are themselves instances. Of metaclasses.
>>> class Foo(object): pass
>>> id(Foo)
142630324
Everything is an object in Python, and they are all either instances of classes or instances of
metaclasses.
Except for type.
type is actually its own metaclass. This is not something you could reproduce in pure Python,
and is done by cheating a little bit at the implementation level.
Secondly, metaclasses are complicated. You may not want to use them for very simple class
alterations. You can change classes by using two different techniques:
8. monkey patching
class decorators
99% of the time you need class alteration, you are better off using these.
But 99% of the time, you don't need class alteration at all.
CREATING SINGLETON IN PYTHON
Method 1: A decorator
def singleton(class_):
instances = {}
def getinstance(*args, **kwargs):
if class_ not in instances:
instances[class_] = class_(*args, **kwargs)
return instances[class_]
return getinstance
@singleton
class MyClass(BaseClass):
pass
Pros
Decorators are additive in a way that is often more intuitive than multiple inheritance.
Cons
While objects created using MyClass() would be true singleton objects, MyClass itself
is a a function, not a class, so you cannot call class methods from it. Also for m =
MyClass(); n = MyClass(); o = type(n)(); then m == n && m != o && n != o
Method 2: A base class
class Singleton(object):
_instance = None
def __new__(class_, *args, **kwargs):
if not isinstance(class_._instance, class_):
class_._instance = object.__new__(class_, *args, **kwargs)
return class_._instance
class MyClass(Singleton, BaseClass):
pass
Pros
It's a true class
Cons
9. Multiple inheritance - eugh! __new__ could be overwritten during inheritance from a
second base class? Have to think more than is necessary.
Method 3: A metaclass
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
#Python2
class MyClass(BaseClass):
__metaclass__ = Singleton
#Python3
class MyClass(BaseClass, metaclass=Singleton):
pass
Pros
It's a true class
Auto-magically covers inheritance
Uses __metaclass__ for its proper purpose (And made me aware of it)
Cons
Are there any?
Method 4: decorator returning a class with the same name
def singleton(class_):
class class_w(class_):
_instance = None
def __new__(class_, *args, **kwargs):
if class_w._instance is None:
class_w._instance = super(class_w, class_).__new__(class_, *args, **kwargs)
class_w._instance._sealed = False
return class_w._instance
def __init__(self, *args, **kwargs):
if self._sealed:
return
super(class_w, self).__init__(*args, **kwargs)
self._sealed = True
class_w.__name__ = class_.__name__
return class_w
@singleton
class MyClass(BaseClass):
pass
10. Pros
It's a true class
Auto-magically covers inheritance
Cons
Is there not an overhead for creating each new class? Here we are creating two
classes for each class we wish to make a singleton. While this is fine in my case I
worry that this might not scale. Of course there is a matter of debate as to whether it
aught to be too easy to scale this pattern...
What is the point of the _sealed attribute
Can't call methods of the same name on base classes using super() because they will
recurse. This means you can't customize __new__ and can't subclass a class that
needs you to call up to __init__.