Python Oddities Explained
I teach Python to teams
My students show me very interesting code
That's odd. What's going on here?
This talk supports Python 3 only
Ask me what oddities Python 2 has
Scope Scares
>>> x = 0
>>> numbers = [1, 1, 2, 3, 5, 8]
>>> for x in numbers:
... y = x**2
...
>>> y
64
>>> x
8
>>> x = 0
>>> numbers = [1, 1, 2, 3, 5, 8]
>>> squares = [x**2 for x in numbers]
>>> x
0 # In Python 2 this was 8
Takeaways
- Reading global variables is perfectly fine
- But don't try to assign to them from a local scope
- List comprehensions have their own scope, loops don't
- Python has no block-level scoping
- Scope matters with assignment, not with mutation
Mesmerizing Mutations
>>> numbers = [1, 2, 3]
>>> numbers2 = numbers
>>> numbers2.append(4)
>>> numbers2
[1, 2, 3, 4]
>>> numbers
[1, 2, 3, 4]
>>> id(numbers)
139670455619848
>>> id(numbers2)
139670455619848
>>> numbers is numbers2
True
>>> x = ([1], [4])
>>> x[0].append(2)
>>> x
([1, 2], [4])
>>> y = x[0]
>>> y.append(3)
>>> x
([1, 2, 3], [4])
>>> x = []
>>> x.append(x)
>>> x
[[...]]
Takeaways
- Lists and dictionaries don't "contain" objects, they contain references (pointers) to objects
- Variables in Python are not buckets that contain things
- Variables are names that point to objects
- Assigning to a variable changes what object it points to
- Mutating an object changes the object itself
- Read Overlooked facts about variables and objects in Python: it's all about pointers
- Watch Facts and Myths about Python names and values
- Watch Names, Objects, and Plummeting From The Cliff
Devious Ducks
>>> duck_list = ['mallard']
>>> duck_list += ('eider', 'Pekin')
>>> duck_list
['mallard', 'eider', 'Pekin']
>>> duck_tuple = ('Muscovy', 'mandarin')
>>> duck_tuple += ['ruddy', 'Indian Runner']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate tuple (not "list") to tuple
>>> ducks = ('Muscovy', 'Indian Runner')
>>> ducks += ('mallard', 'ruddy')
>>> ducks += ['eider', 'Pekin']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate tuple (not "list") to tuple
>>> things = []
>>> things += "duck"
>>> things
['d', 'u', 'c', 'k']
>>> things.extend(' quack')
>>> things
['d', 'u', 'c', 'k', ' ', 'q', 'u', 'a', 'c', 'k']
>>> " ".join(["hello", "there!"])
'hello there!'
>>> " ".join(("hello", "there!"))
'hello there!'
>>> " ".join("hello")
'h e l l o'
>>> dict(('ab', 'cd'))
{'a': 'b', 'c': 'd'}
Duck Typing
Duck Typing
- If it looks like a duck and quacks like a duck, it's a duck
- Don't type check: rely on specific behaviors instead
- "Iterable" and "callable" describe behaviors (not types)
Takeaways
- The list
extendmethod accepts any iterable - The list
+=operator also accepts any iterable of strings - The tuple
+=operator only accepts another tuple - Python thinks in terms of behaviors, not types
- Embrace duck typing by thinking in terms of behaviors: iterable, callable, sequence, mapping, hashable
- More on iterables: Loop Better: a deeper look at iteration
Intriguing In-Place Additions
>>> a = b = (1, 2)
>>> a += (3, 4)
>>> a
(1, 2, 3, 4)
>>> b
(1, 2)
>>> a = a + (3, 4)
>>> a = b = [1, 2]
>>> a += [3, 4]
>>> a
[1, 2, 3, 4]
>>> b
[1, 2, 3, 4]
>>> a = b = [0]
>>> a += b
>>> a, b
([0, 0], [0, 0])
>>> a = b = [0]
>>> a = a + b
>>> a, b
([0, 0, 0, 0], [0, 0])
>>> a = b = [1, 2]
>>> a += [3, 4]
>>> a, b
([1, 2, 3, 4], [1, 2, 3, 4])
>>> c = d = "Python"
>>> c += "!!!" # Same as: c = c + "!!!"
>>> c, d
('Python!!!', 'Python')
>>> c is d
False
>>> a is b
True
>>> x = ([1, 2],)
>>> x[0] += [3, 4]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> x
([1, 2, 3, 4],)
>>> x = ([1, 2],)
>>> x[0] = x[0].__iadd__([3, 4]) # x[0] += [3, 4]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> x[0]
[1, 2, 3, 4]
>>> x[0].__iadd__([5])
[1, 2, 3, 4, 5]
>>> x[0] = [1, 2, 3, 4, 5]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
Takeaways
- In-place addition (
+=) and other "augmented assignment" operations perform assignments - In-place addition calls the
__iadd__method which allows the object to mutate itself if it chooses to do so - In-place addition do the addition "in-place" whenever possible
Closing thoughts
- Understanding how variables, mutability, data structures, operators work is important
- It's important to understand how your language "thinks"
- If it looks like a bug, it might just be a misunderstanding
- Found something odd? Try to learn from it!
#pythonoddity
Recommended resources at
trey.io/oddities
Trey Hunner
Python Team Trainer
>>> '8' < 8
False
>>> '8' < 9
False
>>> '8' < 99999999999999999999999999999999
False
>>> [8] > 8
True
>>> [8] < '8'
True
>>> sorted([str(type(8)), str(type('8')), str(type([8]))])
["<type 'int'>", "<type 'list'>", "<type 'str'>"]
>>> 8 < [8] < '8'
True
>>> class A:
... @property
... def x(self):
... return 'x value'
...
>>> a = A()
>>> a.x
'x value'
>>> a.x = 4
>>> a.x
4
>>> class Cipher:
... alphabet = 'abcdefghijklmnopqrstuvwxyz'
... letter_a = alphabet[0]
... letters = {
... letter: ord(letter) - ord(letter_a)
... for letter in alphabet
... }
...
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in Cipher
File "", line 6, in
NameError: name 'letter_a' is not defined
>>> class Cipher:
... alphabet = 'abcdefghijklmnopqrstuvwxyz'
... letter_a = alphabet[0]
... letters = dict([
... (letter, ord(letter) - ord(letter_a))
... for letter in alphabet
... ])
...
>>> Cipher.letters['a']
0
Scope Silliness
>>> numbers = [1]
>>> class A:
... numbers = [2]
... m = 3
... squares = [n**2 for n in numbers]
...
>>> numbers[0], A.numbers[0], A.m, A.squares[0]
(1, 2, 3, 4)
>>> class A:
... numbers = [2]
... for n in numbers:
... print(n**2)
...
4
>>> A.numbers
[2]
>>> A.n
2
>>> class A:
... numbers = [1, 2]
... m = 3
... squares = [n**m for n in numbers]
... print(squares[0])
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in A
File "<stdin>", line 3, in <listcomp>
NameError: name 'm' is not defined
>>> class A:
... numbers = [1, 2]
... def hi(): print(numbers)
... hi()
...
Traceback (most recent call last):
File "", line 1, in
File "", line 4, in A
File "", line 3, in hi
NameError: name 'numbers' is not defined
>>> VOWELS = "aeiou"
>>> class B:
... VOWELS += "y"
...
>>> B.VOWELS
'aeiouy'
>>> VOWELS
'aeiou'
>>> NUMBERS = [1, 2, 3]
>>> class MyFavoriteClass:
... NUMBERS += [4, 5, 6]
...
>>> MyFavoriteClass.NUMBERS
[1, 2, 3, 4, 5, 6]
>>> NUMBERS
[1, 2, 3, 4, 5, 6]
Illogical Identities
>>> x = 999
>>> y = 999
>>> x == y
True
>>> x is y
False
>>> x = 4
>>> y = 4
>>> x is y
True
lots_of_numbers = range(-1000, 1000)
the_same_numbers = range(-1000, 1000)
same_numbers = (
i
for i, j in zip(lots_of_numbers, the_same_numbers)
if i is j
)
print(*same_numbers, sep=", ")
>>> x = 'a' * 10
>>> y = 'a' * 10
>>> x is y
True
>>> x = 'a' * 1000
>>> y = 'a' * 1000
>>> x is y
False
>>> x = 'a' * 1000
>>> y = 'a' * 1000
>>> m = x[:10]
>>> n = y[:10]
>>> m is n
False
>>> m, n
('aaaaaaaaaa', 'aaaaaaaaaa')
>>> m == n
True
So... should you ever use the is operator?
Yes, when you care about identity, not just equality
def from_node(node):
node_list = []
while node is not None:
node_list.append(node)
node = node.next
return node_list
def is_iterator(iterable):
"""Return True if the given iterable is also an iterator."""
return iter(iterable) is iterable
Takeaways
- Equality and identity are different
- Checking for equality is very common
- Checking for identity is not nearly as common
- Python includes optimizations that make for strange identity patterns with strings and numbers
- Checking for identity when you need equality causes bugs
Mysterious Multiplication
>>> [0] * 3
[0, 0, 0]
>>> [[0] * 3] * 3
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> matrix = [[0] * 3] * 3
>>> matrix[1][1] = 1
>>> matrix
[[0, 1, 0], [0, 1, 0], [0, 1, 0]]
>>> row = [0, 0, 0]
>>> matrix = [row, row, row]
>>> row[1] = 2
>>> matrix
[[0, 2, 0], [0, 2, 0], [0, 2, 0]]
>>> matrix[1][1] = 1
>>> matrix
[[0, 1, 0], [0, 1, 0], [0, 1, 0]]
>>> matrix[0] is matrix[1] is row
True
>>> matrix = [row] * 3
Miscellaneous Mishaps
>>> 'Python' in 'Python' in 'Python'
True
>>> 'Python' in 'Python'
True
>>> True in 'Python'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'in <string>' requires string as left operand, not bool
>>> 'Python' in 'Python' in 'Python'
True
>>> ('Python' in 'Python') in 'Python'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'in <string>' requires string as left operand, not bool
>>> 'Python' in ['Python'] in [['Python']]
True
>>> x = 25
>>> 10 < x < 20
False
>>> """multiline
... strings"""
'multiline\nstrings'
>>> """""strings with
... five quotes around them!"""""
'""strings with\nfive quotes around them!'
>>> """""strings with five
... quotes around them!"""""
'""strings with five\nquotes around them!'