In Python, iterators are object that can be iterated upon. For an object to be an iterator it must have two special methods: __iter__ and __next__.
In Python, iterators are used extensively. While being concealed in plain sight, they are neatly implemented within for loops, comprehensions, generators, etc. A Python iterator object technically needs to implement the iterator protocol, which is made up of the two special methods iter() and next().
If we can create an iterator from an object, it is said to be iterable. The majority of Python’s built-in containers, including list, tuple, string, etc., are iterables.
The iter() function (which in turn calls the iter() method) returns an iterator from them.
We use the next() function to manually iterate through all the items of an iterator. When we read the end and there is no more data to be returned, it will raise the StopIteration Exception. Example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#define a list
heroes = ['Batman', 'Superman', 'Spiderman', 'Ironman']
#get an iterator using iter()
my_iter = iter(heroes)
print(next(my_iter))
Output: 'Batman'
print(next(my_iter))
Output: 'Superman'
# next(obj) is the same as obj.__next__()
print(my_iter.__next_())
Output: 'Spiderman'
print(my_iter.__next__())
Output: 'Ironman'
#Since, iter object has been exhausted, calling next on the iter obj will raise an error - no items left
print(next(my_iter))
Output:
Traceback (most recent call last):
File "<string>", line 24, in <module>
next(my_iter)
StopIteration
Working of for loop for Iterators
As we know, the for loop can iterate over any iterable. Taking a cloaser look at the working of the for loop in Python we get to know that:
1
2
3
4
5
6
7
8
9
10
11
12
#create an iterator object from that iterable
iter_obj = iter(iterable)
#infinite loop
while True:
try:
#get the next item
element = next(iter_obj)
#do something with the lement
except StopIteration:
#if StopIteration is raised, break from loop
break
Internally, the for loop creates an iterator object, iter_obj in this case by callin the iter() on the iterable. Within the for loop is an infinite while loop that continues until the StopIteration condition is met. Inside the while loop, it calles the next() to get the next element and executes the body of the for loop with this value.
Building Custom Iterators
In Python, creating an iterator from scarch is simple, we just need to implement __iter__() and __next__() functions. The iterator object itself is returned by the __iter__() method. Some initialization can be done if necessary. The next element in the series is returned by the __next__() method. It must raise StopIteration error at the conclusion and on subsequent calls.
Example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
class PowTwo:
"""Class to implement an iterator of powers of two"""
def __init__(self, max=0):
self.max = max
def __iter__(self):
self.n=0
return self
def __next__(self):
if self.n <= self.max:
result = 2**self.n
self.n+=1
return result
else:
rase StopIteration
#create an object
numbers = PowTwo(3)
#create in iterable from the object
i = iter(numbers)
#using next to get to the next iterator element
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))
Output:
1
2
4
8
Traceback (most recent call last):
File "/home/bsoyuj/Desktop/Untitled-1.py", line 32, in <module>
print(next(i))
File "<string>", line 18, in __next__
raise StopIteration
StopIteration
Python Infinite Iterators
The item in an iterator object does not necessarily need to be exhausted. Iterators are limitless in number (which never ends). When working with these iterators, we must be cautious. We can build out own infinite iterators. Example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class InfiniteIterator:
"""Infinite Iterator to return all odd numbers""""
def __iter__(self):
self.num=1
return self
def __next__(self):
num = self.num
self.num+=2
return num
a = iter(InfiniteIterator())
print(next(a))
Output: 1
print(next(a))
Output: 3
print(next(a))
Output: 5
Be careful to include a terminating condition, when iterating over these types of infinite iterators.
The advantage of using iterators is that they save resources. Like shown above, we could get all the odd numbers without storing the entire number system in memory. We can have infinite items (theoretically) in finite memory.