How to Read a File Line by Line in Python

John on September 21, 2020

Python is able to read plain text and binary files natively. In this tutorial, we will learn how to open a file in Python and read its contents on a line-by-line basis.

How to Open a File and Read Every Line

To open a file and read every line, use the open() function in reading (r) mode and then use the readlines() method on the file object. readlines() will create a new list containing each line in an element.

apple
strawberry
apricot
orange

# readlines

path = 'text.txt'

with open(path, 'r') as f:
    lines = f.readlines()

matches = []
    
for l in lines:
    if 'app' in l:
        matches.append(l.strip())

print(' '.join(matches))

apple

The example above iterates through the lines list using a for loop. In the for loop there is a conditional to check if the substring ‘app’ exists on the line and if True the line is appended to the matches list. The strip() method is also used to remove \n at the end of each line.

Finally, the matches list is converted into a string using   (space) as the separator before being printed.

Open Large Files in Python with readline()

readline() is often used instead of readlines() when working with large files. The reason for this is it only reads one line of a file at a time rather than opening the entire file at once. This makes it possible to read large amounts of data without exceeding machine resources.

apple
strawberry
apricot
orange

# readline

path = 'text.txt'

file = open('text.txt', 'r')

x = True

while x:
    line = file.readline()
    
    print(line)
    
    if not line:
        x = False

file.close()

apple

strawberry

apricot

orange

The example above creates a new file object, then in a while loop each line of the file is read using the readline() method. A conditional statement is made at the end of each iteration to check if the line is empty (reached the end of the file). If True, x is set to False, ending the while loop. Finally, the file is closed using the close() method.

If we check what the file object is using type() we can see that it is actually a TextIOWrapper, which is a buffered text interface. This is why the next line is read every time readline() is called.

print(type(file))

<class '_io.TextIOWrapper'>

for Loop Through Each Line

Another, cleaner, way to read a file line-by-line is to iterate over the file object using a for loop. This achieves the same result as readline() but takes less code.

# for loop readline

path = 'text.txt'

file = open('text.txt', 'r')

for l in file:
    print(l)

file.close()

strawberry

apricot

orange

Conclusion

You now know three different ways to open a file and read its contents on a line-by-line basis using Python. In most cases, it is fine to open the whole file using the readlines() method. If the file has millions of lines you can use the readline() method to only read one line at a time and not crash your machine.