Dunder Methods in Python: Purpose and Application

Dunder methods (double underscore methods) are special methods in the Python programming language that are surrounded by two underscores at the beginning and end of their names. This naming convention is intended to prevent name conflicts with other user-defined functions.

Each dunder method corresponds to a specific Python language construct that performs a particular data transformation operation.

Here are some commonly used dunder methods:

• __init__(): Initializes an instance of a class, acting as a constructor.
• __repr__(): Returns a representative value of a variable in Python expression format.
• __eq__(): Compares two variables.

Whenever the Python interpreter encounters any syntactic construct, it implicitly calls the corresponding dunder method with all necessary arguments.

For example, when Python encounters the addition symbol in the expression a + b, it implicitly calls the dunder method a.__add__(b), where the addition operation is performed internally.

Thus, dunder methods implement the core mechanics of the Python language. Importantly, these methods are accessible not only to the interpreter but also to ordinary users. Moreover, you can override the implementation of each dunder method within custom classes.

In this guide, we'll explore all existing dunder methods in Python and provide examples.

The demonstrated scripts were run using the Python 3.10.12 interpreter installed on a Hostman cloud server running Ubuntu 22.04.

To run examples from this article, you need to place each script in a separate file with a .py extension (e.g., some_script.py). Then you can execute the file with the following command:

python some_script.py

Creation, Initialization, and Deletion

Creation, initialization, and deletion are the main stages of an object's lifecycle in Python. Each of these stages corresponds to a specific dunder method.

The general algorithm for these methods has specific characteristics:

• Creation: The __new__() method is called with a set of arguments. The first argument is the class of the object (not the object itself). This name is not regulated and can be arbitrary. The remaining arguments are the parameters specified when creating the object in the calling code. The __new__() method must return a class instance — a new object.

• Initialization: Immediately after returning a new object from __new__(), the __init__() method is automatically called. Inside this method, the created object is initialized. The first argument is the object itself, passed as self. The remaining arguments are the parameters specified when creating the object. The first argument name is regulated and must be the keyword self.

• Deletion: Explicit deletion of an object using the del keyword triggers the __del__() method. Its only argument is the object itself, accessed through the self keyword.

Thanks to the ability to override dunder methods responsible for an object's lifecycle, you can create unique implementations for custom classes:

class Initable:
	instances = 0  # class variable, not an object variable

	def __new__(cls, first, second):
		print(cls.instances)
		cls.instances += 1
		return super().__new__(cls)  # call the base class's object creation method with the current class name as an argument

	def __init__(self, first, second):
		self.first = first  # object variable
		self.second = second  # another object variable

	def __del__(self):
		print("Annihilated!")

inited = Initable("Initialized", 13)  # output: 0

print(inited.first)  # output: Initialized
print(inited.second)  # output: 13

del inited  # output: Annihilated!

Thanks to these hook-like methods, you can manage not only the internal state of an object but also external resources.

Comparison

Objects created in Python can be compared with one another, yielding either a positive or negative result. Each comparison operator is associated with a corresponding dunder method.

In some cases, Python provides multiple syntactic constructs for the same comparison operations. We can replace each of these operations by the corresponding dunder method:

a = 5
b = 6
c = "This is a regular string"

print(a == b)  # Output: False
print(a is b)  # Output: False
print(a.__eq__(b))  # Output: False

print(a != b)  # Output: True
print(a is not b)  # Output: True
print(a.__ne__(b))  # Output: True
print(not a.__eq__(b))  # Output: True

print(a > b)  # Output: False
print(a < b)  # Output: True
print(a >= b)  # Output: False
print(a <= b)  # Output: True

print(a.__gt__(b))  # Output: False
print(a.__lt__(b))  # Output: True
print(a.__ge__(b))  # Output: False
print(a.__le__(b))  # Output: True

print(hash(a))  # Output: 5
print(a.__hash__())  # Output: 5

print(c.__hash__())  # Output: 1745008793

The __ne__() method returns the inverted result of the __eq__() method. Because of this, there's often no need to redefine __ne__() since the primary comparison logic is usually implemented in __eq__().

Additionally, some comparison operations implicitly performed by Python when manipulating list elements require hash computation. For this purpose, Python provides the special dunder method __hash__().

By default, any user-defined class already implements the methods __eq__(), __ne__(), and __hash__():

class Comparable:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

c1 = Comparable(4, 3)
c2 = Comparable(7, 9)

print(c1 == c1)  # Output: True
print(c1 != c1)  # Output: False

print(c1 == c2)  # Output: False
print(c1 != c2)  # Output: True

print(c1.__hash__())  # Example output: -2146408067
print(c2.__hash__())  # Example output: 1076316

In this case, the default __eq__() method compares instances without considering their internal variables defined in the __init__() constructor. The same applies to the __hash__() method, whose results vary between calls.

Python's mechanics are designed such that overriding the __eq__() method automatically removes the default __hash__() method:

class Comparable:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

	def __eq__(self, other):
		if isinstance(other, self.__class__):
			return self.value1 == other.value1 and self.value2 == other.value2
		return False

c1 = Comparable(4, 3)
c2 = Comparable(7, 9)

print(c1 == c1)  # Output: True
print(c1 != c1)  # Output: False

print(c1 == c2)  # Output: False
print(c1 != c2)  # Output: True

print(c1.__hash__())  # ERROR (method not defined)
print(c2.__hash__())  # ERROR (method not defined)

Therefore, overriding the __eq__() method requires also overriding the __hash__() method with a new hashing algorithm:

class Comparable:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

	def __eq__(self, other):
		if isinstance(other, self.__class__):
			return self.value1 == other.value1 and self.value2 == other.value2
		return False

	def __ne__(self, other):
		return not self.__eq__(other)

	def __gt__(self, other):
		return self.value1 + self.value2 > other.value1 + other.value2

	def __lt__(self, other):
		return not self.__gt__(other)

	def __ge__(self, other):
		return self.value1 + self.value2 >= other.value1 + other.value2

	def __le__(self, other):
		return self.value1 + self.value2 <= other.value1 + other.value2

	def __hash__(self):
		return hash((self.value1, self.value2))  # Returns the hash of a tuple of two numbers

c1 = Comparable(4, 3)
c2 = Comparable(7, 9)

print(c1 == c1)  # Output: True
print(c1 != c1)  # Output: False

print(c1 == c2)  # Output: False
print(c1 != c2)  # Output: True

print(c1 > c2)  # Output: False
print(c1 < c2)  # Output: True

print(c1 >= c2)  # Output: False
print(c1 <= c2)  # Output: True

print(c1.__hash__())  # Output: -1441980059
print(c2.__hash__())  # Output: -2113571365

Thus, by overriding comparison methods, you can use standard syntactic constructs for custom classes, similar to built-in data types, regardless of their internal complexity.

Conversion in Python

In Python, we can convert all built-in types from one to another. Similar conversions can be added to custom classes, considering the specifics of their internal implementation.

By default, we can only convert a custom class to a few basic types:

class Convertible:
    def __init__(self, value1, value2):
        self.value1 = value1
        self.value2 = value2

some_variable = Convertible(4, 3)

print(str(some_variable))  # Example output: <__main__.Convertible object at 0x1229620>
print(bool(some_variable))  # Output: True

However, by overriding the corresponding dunder methods, you can implement conversions from a custom class to any built-in data type:

class Convertible:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

	def __str__(self):
		return str(self.value1) + str(self.value2)

	def __bool__(self):
		return self.value1 == self.value2

	def __int__(self):
		return self.value1 + self.value2

	def __float__(self):
		return float(self.value1) + float(self.value2)

	def __bytes__(self):
		return bytes(self.value1) + bytes(self.value2)

	def __complex__(self):
		return complex(self.value1) + complex(self.value2)

someVariable = Convertible(4, 3)

print(str(someVariable)) # output: 43
print(bool(someVariable)) # output: False
print(int(someVariable)) # output: 7
print(float(someVariable)) # output: 7.0
print(bytes(someVariable)) # output: b'\x00\x00\x00\x00\x00\x00\x00'
print(complex(someVariable)) # output: (7+0j)

Thus, implementing dunder methods for conversion allows objects of custom classes to behave like built-in data types, enhancing their completeness and versatility.

Element Management in Python

Just like lists, we can make any custom class in Python iterable. Python provides corresponding dunder methods for retrieving and manipulating elements.

Even though the internal implementation of an iterable custom class can vary, element management is handled using Python's standard container interface rather than custom methods.

class Iterable:
	def __init__(self, e1, e2, e3, e4):
		self.e1 = e1
		self.e2 = e2
		self.e3 = e3
		self.e4 = e4
		self.index = 0

	def __len__(self):
		len = 0
		if self.e1: len += 1
		if self.e2: len += 1
		if self.e3: len += 1
		if self.e4: len += 1
		return len

	def __iter__(self):
		for i in range(0, self.__len__() + 1):
			if i == 0: yield self.e1
			if i == 1: yield self.e2
			if i == 2: yield self.e3
			if i == 3: yield self.e4

	def __getitem__(self, item):
		if item == 0: return self.e1
		elif item == 1: return self.e2
		elif item == 2: return self.e3
		elif item == 3: return self.e4
		else: raise Exception("Out of range")

	def __setitem__(self, item, value):
		if item == 0: self.e1 = value
		elif item == 1: self.e2 = value
		elif item == 2: self.e3 = value
		elif item == 3: self.e4 = value
		else: raise Exception("Out of range")

	def __delitem__(self, item):
		if item == 0: self.e1 = None
		elif item == 1: self.e2 = None
		elif item == 2: self.e3 = None
		elif item == 3: self.e4 = None
		else: raise Exception("Out of range")

	def __contains__(self, item):
		if self.e1 == item: return true
		elif self.e2 == item: return True
		elif self.e3 == item: return True
		elif self.e4 == item: return True
		else: return False

	def __reversed__(self):
		return Iterable(self.e4, self.e3, self.e2, self.e1)

	def __next__(self):
		if self.index >=4: self.index = 0
		if self.index == 0: element = self.e1
		if self.index == 1: element = self.e2
		if self.index == 2: element = self.e3
		if self.index == 3: element = self.e4
		self.index += 1
		return element

someContainer = Iterable(-2, 54, 6, 13)

print(someContainer.__len__()) # output: 4

print(someContainer[0]) # output: -2
print(someContainer[1]) # output: 54
print(someContainer[2]) # output: 6
print(someContainer[3]) # output: 13

someContainer[2] = 117
del someContainer[0]

print(someContainer[2]) # output: 117

for element in someContainer:
	print(element) # output: None, 54, 117, 13

print(117 in someContainer) # output: True

someContainerReversed = someContainer.__reversed__()

for element in someContainerReversed:
	print(element) # output: 13, 117, 54, None

print(someContainer.__next__()) # output: None
print(someContainer.__next__()) # output: 54
print(someContainer.__next__()) # output: 117
print(someContainer.__next__()) # output: 13
print(someContainer.__next__()) # output: None

It’s important to understand the difference between the __iter__() and __next__() methods, which facilitate object iteration.

• __iter__() iterates the object at a given point.
• __next__() returns an element considering an internal index.

A particularly interesting dunder method is __missing__(), which is only relevant in custom classes inherited from the base dictionary type dict.

This method allows you to override the default dict behavior when attempting to retrieve a non-existent element:

class dict2(dict):
	def __missing__(self, item):
		return "Sorry but I don’t exist..."

someDictionary = dict2(item1=10, item2=20, item3=30)

print(someDictionary["item1"]) # output: 10
print(someDictionary["item2"]) # output: 20
print(someDictionary["item3"]) # output: 30
print(someDictionary["item4"]) # output: Sorry but I don’t exist...

Arithmetic Operations

Arithmetic operations are the most common type of data manipulation. Python provides corresponding syntactic constructs for performing addition, subtraction, multiplication, and division.

Most often, left-handed methods are used, which perform calculations on behalf of the left operand.

If the right operand does not know how to perform the operation, Python automatically calls a right-handed method, which calculates the value on behalf of the right operand.

However, in this case, the operands must be of different types.

It is also possible to override in-place arithmetic operations. In this case, dunder methods do not return a new value but modify the existing variables of the left operand.

By overriding these corresponding dunder methods, you can define specific behaviors for your custom class during arithmetic operations:

class Arithmetic:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

	def __add__(self, other):
		return Arithmetic(self.value1 + other.value1, self.value2 + other.value2)

	def __radd__(self, other):
		return Arithmetic(other + self.value1, other + self.value2)

	def __iadd__(self, other):
		self.value1 += other.value1
		self.value2 += other.value2
		return self

	def __sub__(self, other):
		return Arithmetic(self.value1 - other.value1, self.value2 - other.value2)

	def __rsub__(self, other):
		return Arithmetic(other - self.value1, other - self.value2)

	def __isub__(self, other):
		self.value1 -= other.value1
		self.value2 -= other.value2
		return self

	def __mul__(self, other):
		return Arithmetic(self.value1 * other.value1, self.value2 * other.value2)

	def __rmul__(self, other):
		return Arithmetic(other * self.value1, other * self.value2)

	def __imul__(self, other):
		self.value1 *= other.value1
		self.value2 *= other.value2
		return self

	def __truediv__(self, other):
		return Arithmetic(self.value1 / other.value1, self.value2 / other.value2)

	def __rtruediv__(self, other):
		return Arithmetic(other / self.value1, other / self.value2)

	def __itruediv__(self, other):
		self.value1 /= other.value1
		self.value2 /= other.value2
		return self

	def __mod__(self, other):
		return Arithmetic(self.value1 % other.value1, self.value2 % other.value2)

	def __rmod__(self, other):
		return Arithmetic(other % self.value1, other % self.value2)

	def __imod__(self, other):
		self.value1 %= other.value1
		self.value2 %= other.value2
		return self

	def __floordiv__(self, other):
		return Arithmetic(self.value1 // other.value1, self.value2 // other.value2)

	def __rfloordiv__(self, other):
		return Arithmetic(other // self.value1, other // self.value2)

	def __ifloordiv__(self, other):
		self.value1 //= other.value1
		self.value2 //= other.value2
		return self

	def __pow__(self, other):
		return Arithmetic(self.value1 ** other.value1, self.value2 ** other.value2)

	def __rpow__(self, other):
		return Arithmetic(other ** self.value1, other ** self.value2)

	def __ipow__(self, other):
		self.value1 **= other.value1
		self.value2 **= other.value2
		return self

a1 = Arithmetic(4, 6)
a2 = Arithmetic(10, 3)

add = a1 + a2
sub = a1 - a2
mul = a1 * a2
truediv = a1 / a2
mod = a1 % a2
floordiv = a1 // a2
pow = a1 ** a2

radd = 50 + a1
rsub = 50 - a2
rmul = 50 * a1
rtruediv = 50 / a2
rmod = 50 % a1
rfloordiv = 50 // a2
rpow = 50 ** a2

a1 -= a2
a1 *= a2
a1 /= a2
a1 %= a2
a1 //= a2
a1 **= a2

print(add.value1, add.value2) # output: 14 9
print(sub.value1, sub.value2) # output: -6 3
print(mul.value1, mul.value2) # output: 40 18
print(truediv.value1, truediv.value2) # output: 0.4 2.0
print(mod.value1, mod.value2) # output: 4 0
print(floordiv.value1, floordiv.value2) # output: 0 2
print(pow.value1, pow.value2) # output: 1048576 216

print(radd.value1, radd.value2) # output: 54 56
print(rsub.value1, rsub.value2) # output: 40 47
print(rmul.value1, rmul.value2) # output: 200 300
print(rtruediv.value1, rtruediv.value2) # output: 5.0 16.666666666666668
print(rmod.value1, rmod.value2) # output: 2 2
print(rfloordiv.value1, rfloordiv.value2) # output: 5 16
print(rpow.value1, rpow.value2) # output: 97656250000000000 125000

In real-world scenarios, arithmetic dunder methods are among the most frequently overridden. Therefore, it is good practice to implement both left-handed and right-handed methods simultaneously.

Bitwise Operations

In addition to standard mathematical operations, Python allows you to override the behavior of custom classes during bitwise transformations.

Similar to arithmetic operations, bitwise transformations can be performed on behalf of the right operand.

Naturally, bitwise operations can also be performed in-place, modifying the left operand instead of returning a new object.

By overriding these dunder methods, any custom class can perform familiar bitwise operations on its contents seamlessly.

class Bitable:
	def __init__(self, value1, value2):
		self.value1 = value1
		self.value2 = value2

	def __and__(self, other):
		return Bitable(self.value1 & other.value1, self.value2 & other.value2)

	def __rand__(self, other):
		return Bitable(other & self.value1, other & self.value2)

	def __iand__(self, other):
		self.value1 &= other.value1
		self.value2 &= other.value2
		return self

	def __or__(self, other):
		return Bitable(self.value1 | other.value1, self.value2 | other.value2)

	def __ror__(self, other):
		return Bitable(other | self.value1, other | self.value2)

	def __ior__(self, other):
		self.value1 |= other.value1
		self.value2 |= other.value2
		return self

	def __xor__(self, other):
		return Bitable(self.value1 ^ other.value1, self.value2 ^ other.value2)

	def __rxor__(self, other):
		return Bitable(other ^ self.value1, other ^ self.value2)

	def __ixor__(self, other):
		self.value1 |= other.value1
		self.value2 |= other.value2
		return self

	def __rshift__(self, other):
		return Bitable(self.value1 >> other.value1, self.value2 >> other.value2)

	def __rrshift__(self, other):
		return Bitable(other >> self.value1, other >> self.value2)

	def __irshift__(self, other):
		self.value1 >>= other.value1
		self.value2 >>= other.value2
		return self

	def __lshift__(self, other):
		return Bitable(self.value1 << other.value1, self.value2 << other.value2)

	def __rlshift__(self, other):
		return Bitable(other << self.value1, other << self.value2)

	def __ilshift__(self, other):
		self.value1 <<= other.value1
		self.value2 <<= other.value2
		return self

b1 = Bitable(5, 3)
b2 = Bitable(7, 2)

resultAnd = b1 & b2
resultOr = b1 | b2
resultXor = b1 ^ b2
resultRshift = b1 >> b2
resultLshift = b1 << b2

resultRand = 50 & b1
resultRor = 50 | b2
resultRxor = 50 ^ b1
resultRrshift = 50 >> b2
resultRlshift = 50 << b1

b1 &= b2
b1 |= b2
b1 ^= b2
b1 >>= b2
b1 <<= b2

print(resultAnd.value1, resultAnd.value2) # output: 5 2
print(resultOr.value1, resultAnd.value2) # output: 7 2
print(resultXor.value1, resultAnd.value2) # output: 2 2
print(resultRshift.value1, resultAnd.value2) # output: 0 2
print(resultLshift.value1, resultAnd.value2) # output: 640 2

print(resultRand.value1, resultRand.value2) # output: 0 2
print(resultRor.value1, resultRor.value2) # output: 55 50
print(resultRxor.value1, resultRxor.value2) # output: 55 49
print(resultRrshift.value1, resultRrshift.value2) # output: 0 12
print(resultRlshift.value1, resultRlshift.value2) # output: 1600 400

In addition to operations involving two operands, Python provides dunder methods for bitwise transformations involving a single operand.

The + operator typically does not affect the value of the variable. Many classes override this method to perform alternative transformations.

Object Information Extraction

Python offers several dunder methods to retrieve additional information about an object.

• __str__() returns a user-friendly string representation of the variable’s value.
• __repr__() returns a more detailed and often code-like representation of the variable, suitable for recreating the original variable via eval().

So, it is important for a custom class to be able to provide additional information about itself.

class Human:
	def __init__(self, name, age):
		self.name = name
		self.age = age

	def __str__(self):
		return str(self.name + " (" + str(self.age) + " years old)")

	def __repr__(self):
		return "Human(" + repr(self.name) + ", " + str(self.age) + ")"

someHuman = Human("John", 35)
someOtherHuman = eval(repr(someHuman))

print(str(someHuman)) # output: John (35 years old)
print(repr(someHuman)) # output: Human('John', 35)

print(str(someOtherHuman)) # output: John (35 years old)
print(repr(someOtherHuman)) # output: Human('John', 35)

Conclusion

A distinctive feature of Python dunder methods is using two underscore characters at the beginning and end of the name, which prevents naming conflicts with other user-defined functions.

Unlike regular control methods, dunder methods allow you to define unique behavior for a custom class when using standard Python operators responsible for:

• Arithmetic operations
• Iteration and access to elements
• Creation, initialization, and deletion of objects

Additional dunder attributes provide auxiliary information about Python program entities, which can simplify the implementation of custom classes.