How do you filter a list based on a condition in python?

Python Filter List By String Length

Given a list of strings.

How to obtain all elements that have more than x characters? In other words: how to filter a list by string length?

coders = ['Ann', 'Alice', 'Frank', 'Pit']
filtered = [x for x in coders if len[x]>3]
print[filtered]
# ['Alice', 'Frank']

The list comprehension statement [x for x in coders if len[x]>3] filters all strings that have more than three characters.

Python Filter List None

How to remove all None values from a list?

For example, you have the list ['Alice', None, 'Ann', None, None, 'Bob'] and you want the list ['Alice', 'Ann', 'Bob']. How do you do this?

coders = ['Alice', None, 'Ann', None, None, 'Bob']
filtered = [x for x in coders if x]
print[filtered]
# ['Alice', 'Ann', 'Bob']

In Python, each element has an associated Boolean value so you can use any Python object as a condition. The value None is associated with Boolean value False.

Python Filter JSON List

Problem: Say, you’ve got a JSON list object. You want to filter the list based on an attribute. How to accomplish that?

Example: Given the following JSON list.

json = [
    {
        "user": "alice",
        "type": "free"
    },
    {
        "user": "ann",
        "type": "paid"
    },
    {
        "user": "bob",
        "type": "paid"
    }
]

You want to find all users that have a 'paid' account type:

[
    {
        "user": "ann",
        "type": "paid"
    },
    {
        "user": "bob",
        "type": "paid"
    }
]

Solution: Use list comprehension [x for x in json if x['type']=='paid'] to filter the list and obtain a new JSON list with the objects that pass the filter.

json = [
    {
        "user": "alice",
        "type": "free"
    },
    {
        "user": "ann",
        "type": "paid"
    },
    {
        "user": "bob",
        "type": "paid"
    }
]

filtered = [x for x in json if x['type']=='paid']
print[filtered]
# [{'user': 'ann', 'type': 'paid'},
#  {'user': 'bob', 'type': 'paid'}]

Only Ann and Bob have a paid account and pass the test x['type']=='paid'.

Python Filter List One Line

Want to filter your list by a given condition in one line of code? Use the list comprehension statement [x for x in list if condition] where the condition part can be any Boolean expression on x. This one-liner returns a new list object with all elements that pass the filtering “test”.

Here’s an example:

lst = ['Alice', 3, 5, 'Bob', 10]

# ONE-LINER:
f = [x for x in lst if type[x]==str]

print[f]
# ['Alice', 'Bob']

The one-liner filters all elements in the list and checks whether they are of type string. If they are, they pass the test and are included in the new list.

If you like one-liners, you’ll love my Python One-Liner book [NoStarch Press 2020]. It shows you exactly how to write Pythonic code and compress your thinking and coding to the most minimalistic form.

Python Filter List Efficiently: filter[] vs List Comprehension

[Spoiler] Which is faster to filter a list: filter[] vs list comprehension? For large lists with one million elements, filtering lists with list comprehension is 40% faster than the built-in filter[] method.

To answer this question, I’ve written a short script that tests the runtime performance of filtering large lists of increasing sizes using the filter[] and the list comprehension methods.

My thesis is that the list comprehension method should be slightly faster for larger list sizes because it leverages the efficient cPython implementation of list comprehension and doesn’t need to call an extra function.

I used my notebook with an Intel[R] Core[TM] i7-8565U 1.8GHz processor [with Turbo Boost up to 4.6 GHz] and 8 GB of RAM.

Then, I created 100 lists with both methods with sizes ranging from 10,000 elements to 1,000,000 elements. As elements, I simply incremented integer numbers by one starting from 0.

Here’s the code I used to measure and plot the results: which method is faster—filter[] or list comprehension?

import time


# Compare runtime of both methods
list_sizes = [i * 10000 for i in range[100]]
filter_runtimes = []
list_comp_runtimes = []

for size in list_sizes:

    lst = list[range[size]]
    
    # Get time stamps
    time_0 = time.time[]
    list[filter[lambda x: x%2, lst]]
    time_1 = time.time[]
    [x for x in lst if x%2]
    time_2 = time.time[]

    # Calculate runtimes
    filter_runtimes.append[[size, time_1 - time_0]]
    list_comp_runtimes.append[[size, time_2 - time_1]]


# Plot everything
import matplotlib.pyplot as plt
import numpy as np

f_r = np.array[filter_runtimes]
l_r = np.array[list_comp_runtimes]

print[filter_runtimes]
print[list_comp_runtimes]

plt.plot[f_r[:,0], f_r[:,1], label='filter[]']
plt.plot[l_r[:,0], l_r[:,1], label='list comprehension']

plt.xlabel['list size']
plt.ylabel['runtime [seconds]']

plt.legend[]
plt.savefig['filter_list_comp.jpg']
plt.show[]

The code compares the runtimes of the filter[] function and the list comprehension variant to filter a list. Note that the filter[] function returns a filter object, so you need to convert it to a list using the list[] constructor.

Here’s the resulting plot that compares the runtime of the two methods.

On the x axis, you can see the list size from 0 to 1,000,000 elements. On the y axis, you can see the runtime in seconds needed to execute the respective functions.

The resulting plot shows that both methods are extremely fast for a few tens of thousands of elements. In fact, they are so fast that the time[] function of the time module cannot capture the elapsed time.

But as you increase the size of the lists to hundreds of thousands of elements, the list comprehension method starts to win:

For large lists with one million elements, filtering lists with list comprehension is 40% faster than the built-in filter[] method.

The reason is the efficient implementation of the list comprehension statement. An interesting observation is the following though. If you don’t convert the filter function to a list, you get the following result:

Suddenly the filter[] function has constant runtime of close to 0 seconds—no matter how many elements are in the list. Why is this happening?

The explanation is simple: the filter function returns an iterator, not a list. The iterator doesn’t need to compute a single element until it is requested to compute the next[] element.

So, the filter[] function computes the next element only if it is required to do so. Only if you convert it to a list, it must compute all values. Otherwise, it doesn’t actually compute a single value beforehand.

Programming Humor – Python

Where to Go From Here

This tutorial has shown you the ins and outs of the filter[] function in Python and compared it against the list comprehension way of filtering: [x for x in list if condition]. You’ve seen that the latter is not only more readable and more Pythonic, but also faster. So take the list comprehension approach to filter lists!

If you love coding and you want to do this full-time from the comfort of your own home, you’re in luck:

I’ve created a free webinar that shows you how I started as a Python freelancer after my computer science studies working from home [and seeing my kids grow up] while earning a full-time income working only part-time hours.

Webinar: How to Become Six-Figure Python Freelancer?

Join 21,419 ambitious Python coders. It’s fun!

While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students.

To help students reach higher levels of Python success, he founded the programming education website Finxter.com. He’s author of the popular programming book Python One-Liners [NoStarch 2020], coauthor of the Coffee Break Python series of self-published books, computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide.

His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here.

How do you filter a list by condition in Python?

How do you filter the elements based on a function in a Python list?

How do you filter data from a list?

How do you check if every element of a list satisfies a condition in Python?