How do you filter a list based on a condition in python?
|
How can you filter a list in Python using an arbitrary condition? Show
The most Pythonic and most performant way is to use list comprehension How to Filter a List in Python?
Filter with List ComprehensionThe most Pythonic way of filtering a list—in my opinion—is the list comprehension statement For
example, if you want to filter all elements that are smaller than, say, 10, you’d use the list comprehension statement Here are three examples of filtering a list:
lst = [8, 2, 6, 4, 3, 1] # Filter all elements <8 small = [x for x in lst if x<8] print(small) # Filter all even elements even = [x for x in lst if x%2==0] print(even) # Filter all odd elements odd = [x for x in lst if x%2] print(odd) The output is: # Elements <8 [2, 6, 4, 3, 1] # Even Elements [8, 2, 6, 4] # Odd Elements [3, 1] This is the most efficient way of filtering a list and it’s also the most Pythonic one. If you look for alternatives though, keep reading because I’ll explain to you each and every nuance of filtering lists in Python in this comprehensive guide. The You can use the The syntax of the lambda function is Related Tutorial: For more information, see my detailed blog article about the lambda function. lst = [8, 2, 6, 4, 3, 1] # Filter all elements <8 small = filter(lambda x: x<8, lst) print(list(small)) # Filter all even elements even = filter(lambda x: x%2==0, lst) print(list(even)) # Filter all odd elements odd = filter(lambda x: x%2, lst) print(list(odd)) The output is: # Elements <8 [2, 6, 4, 3, 1] # Even Elements [8, 2, 6, 4] # Odd Elements [3, 1] The Related article:
Filter a Python List with map()I just add this option because people are still trying to use the 🛑 This is clearly the wrong way of doing it. The reason is that the Therefore, you need to have an extra step of filtering out all elements (for example, by using list comprehension). But if you’re ready to take this extra step, you could also use list comprehension for filtering in the first place. Here’s what I mean: lst = [8, 2, 6, 4, 3, 1] # Filter all elements <8 small = list(map(lambda x: x if x<8 else None, lst)) small = [x for x in small if x!=None] print(small) # Filter all even elements even = list(map(lambda x: x if x%2==0 else None, lst)) even = [x for x in even if x!=None] print(even) # Filter all odd elements odd = list(map(lambda x: x if x%2 else None, lst)) odd = [x for x in odd if x!=None] print(odd) The output is again the same: [2, 6, 4, 3, 1] [8, 2, 6, 4] [3, 1] But the method of getting this output is clearly ineffective and not readable. Related articles:
Python Filter List GeneratorA generator expression creates an iterator over a sequence of values. It works just like list comprehension—but without creating a list data type. This is a bit more efficient. You can use generator expressions in any function call that requires an iterable as input. For example, if you want to calculate the sum of all values that meet a certain condition in a list.
Here’s a code example that shows how to sum over all integer values in a list (and ignoring the rest) using a generator expression to filter out those non-integers: lst = [6, 8, 2, 8, 'Alice'] print(sum(x for x in lst if type(x) == int)) # 24 You check the Python Filter List with ConditionYou can define any complicated condition on a list element to decide whether to filter it out or not. Just create your own function (e.g., Here’s a code example: def condition(x):
'''Define your arbitrarily
complicated condition here'''
return x<10 and x>0
lst = [11, 14, 3, 0, -1, -3]
# Filter out all elements that do
# not meet condition
filtered = [x for x in lst if condition(x)]
print(filtered)
# [3]All elements that are smaller than 10 and larger than 0 are included in the filtered list. Thus, only one element 3 remains. Python Filter List Multiple ConditionsThe same applies if you want to combine multiple conditions. Say, you want to filter out all elements You can define any complicated condition on a list element to decide whether to filter it out or not. To do this, create a
function (e.g., Here’s the same code example as before: def condition(x):
'''Define your arbitrarily
complicated condition here'''
return x<10 and x>0
lst = [11, 14, 3, 0, -1, -3]
# Filter out all elements that do
# not meet condition
filtered = [x for x in lst if condition(x)]
print(filtered)
# [3]All elements that are smaller than 10 and larger than 0 are included in the filtered list. Thus, only one element 3 remains. Python Filter List with RegexProblem: Given a list of strings. How can you filter those that match a certain regular expression? Example: Say, you’ve got a list Solution: Use the list comprehension filtering framework import re # Define the list and the regex pattern to match customers = ['Alice', 'Anne', 'Ann', 'Tom'] pattern = 'A.*e' # Filter out all elements that match the pattern filtered = [x for x in customers if re.match(pattern, x)] print(filtered) # ['Alice', 'Anne'] You use the In Python, any match object evaluates to If you need to refresh your basic understanding of the Related articles:
How to Filter a Python List of Lists?Short answer: To filter a list of lists for a condition on the inner lists, use the list comprehension statement Feel free to also watch my video on the topic: How to Filter a List in Python? Lists belong to the most important data structures in Python—every master coder knows them by heart! Surprisingly, even intermediate coders don’t know the best way to filter a list—let alone a list of lists in Python. This tutorial shows you how to do the latter! Problem: Say, you’ve got a list of lists. You want to filter the list of lists so that only those inner lists remain that satisfy a certain condition. The condition is a function of the inner list—such as the average or sum of the inner list elements.
 Example: Given the following list of lists with weekly temperature measurements per week—and one inner list per week. # Measurements of a temperature sensor (7 per week)
temperature = [[10, 8, 9, 12, 13, 7, 8], # week 1
[9, 9, 5, 6, 6, 9, 11], # week 2
[10, 8, 8, 5, 6, 3, 1]] # week 3How to filter out the colder weeks with average temperature value <8? This is the output you desire: print(cold_weeks) # [[9, 9, 5, 6, 6, 9, 11], [10, 8, 8, 5, 6, 3, 1]] There are two semantically equivalent methods to achieve this: list comprehension and the
Related articles:
Python Filter List of StringsProblem: Given a list of strings and a query string. How can you filter those that contain the query string? Example: Say, you’ve got a list Solution: Use the list comprehension filtering framework import re # Define the list customers = ['Alice', 'Anne', 'Ann', 'Tom'] # Filter out all elements that contain 'An' filtered = [x for x in customers if 'An' in x] print(filtered) # ['Anne', 'Ann'] You use the basic string membership operation “in” to check if an element passes the filter or not. Python Filter List Remove Empty StringProblem: Given a list of strings. How can you remove all empty strings? Example: Say, you’ve got a list Solution: Use the list comprehension filtering framework import re # Define the list customers = ['Alice', 'Anne', '', 'Ann', '', 'Tom'] # Filter out all elements that contain 'An' filtered = [x for x in customers if x] print(filtered) # ['Alice', 'Anne', 'Ann', 'Tom'] You use the property that Python assigns
the Boolean value Python Filter List endswith() and startswith()Problem: Given a list of strings. How can you filter those that starts with another string (or end with another string)? In other words, you want to get all strings that have another string as prefix or suffix. Example: Say, you’ve got a list Solution: Use the list comprehension filtering framework If you want to check strings that end with another string, you can use the import re
# Define the list
customers = ['Alice', 'Anne', 'Ann', 'Tom']
# Filter out all elements that start with 'An'
filtered = [x for x in customers if x.startswith('An')]
print(filtered)
# ['Anne', 'Ann']
# Filter out all elements that end with 'e'
filtered = [x for x in customers if x.endswith('e')]
print(filtered)
# ['Alice', 'Anne']
You use the Python Filter List with LambdaThe To define this function, you can use the Here’s the code that shows how to filter a list using the lambda function to filter a list and returning only the odd values in the list: # Create the list lst = [1, 2, 3, 4] # Get all odd values print(list(filter(lambda x: x%2, lst))) # [1, 3] The lambda function Thus, all odd elements pass the test. Python Filter List with Another ListProblem: Given a list of
values Example: Here are two example lists: lst = [1, 2, 3, 4] filter_lst = [True, False, False, True] And you want to obtain this list: [1, 4] Solution: Use a simple list comprehension statement Here’s the code: lst = [1, 2, 3, 4] filter_lst = [True, False, False, True] res = [lst[i] for i in range(len(lst)) if filter_lst[i]] print(res) # [1, 4] The Boolean list serves as a “mask” that determines which element passes the filter and which does not. An alternative is to use the lst = [1, 2, 3, 4] filter_lst = [True, False, False, True] res = [x for (x, boo) in zip(lst, filter_lst) if boo] print(res) # [1, 4] Related: Do you need to brush up your Python Filter List with IndexProblem: Given a list of values and a list of indices. How to filter all elements with indices in the second list? Example: You’ve list Solution: Go over all indices in the second list and include the corresponding list elements using a simple list comprehension statement lst = ['Alice', 'Bob', 'Ann', 'Frank'] indices = [1, 2] res = [lst[i] for i in indices] print(res) # ['Bob', 'Ann'] Only two elements with indices 1 and 2 pass the filter. Python Filter List of DictionariesProblem: Given a list of dictionaries. Each dictionary consists of one or more (key, value) pairs. You want to filter them by value of a particular dictionary key (attribute). How do you do this? Minimal Example: Consider the following example where you’ve three user dictionaries with Here’s what you try to accomplish: users = [{'username': 'alice', 'age': 23, 'play_time': 101},
{'username': 'bob', 'age': 31, 'play_time': 88},
{'username': 'ann', 'age': 25, 'play_time': 121},]
superplayers = # Filtering Magic Here
print(superplayers)
The output should look like this where the [{'username': 'alice', 'age': 23, 'play_time': 101},
{'username': 'ann', 'age': 25, 'play_time': 121}]Solution: Use list comprehension Here’s the code that shows you how to filter out all user dictionaries that don’t meet the condition of having played at least 100 hours. users = [{'username': 'alice', 'age': 23, 'play_time': 101},
{'username': 'bob', 'age': 31, 'play_time': 88},
{'username': 'ann', 'age': 25, 'play_time': 121},]
superplayers = [user for user in users if user['play_time']>100]
print(superplayers)The output is the filtered list of dictionaries that meet the condition: [{'username': 'alice', 'age': 23, 'play_time': 101},
{'username': 'ann', 'age': 25, 'play_time': 121}]Related articles on the Finxter blog:
Python Filter List Unique – Remove DuplicatesHow to remove all duplicates of a given value in the list? The naive approach is to go over each element and check whether this element already exists in the list. If so, remove it. However, this takes a few lines of code. A shorter and more concise way is to create a dictionary out of the elements in the list. Each list element becomes a new key to the dictionary. All elements that occur multiple times will be assigned to the same key. The dictionary contains only unique keys—there cannot be multiple equal keys. As dictionary values, you simply take dummy values (per default). Related blog articles:
Then, you simply convert the dictionary back to a list throwing away the dummy values. As the dictionary keys stay in the same order, you don’t lose the order information of the original list elements. Here’s the code: >>> lst = [1, 1, 1, 3, 2, 5, 5, 2]
>>> dic = dict.fromkeys(lst)
>>> dic
{1: None, 3: None, 2: None, 5: None}
>>> duplicate_free = list(dic)
>>> duplicate_free
[1, 3, 2, 5]Python Filter List RangeFilter all elements in a list that fall into a range of values between given lst = [3, 10, 3, 2, 5, 1, 11] start, stop = 2, 9 filtered_lst = [x for x in lst if x>=start and x<=stop] print(filtered_lst) # [3, 3, 2, 5] You use the condition Python Filter List Greater Than and Less ThanFilter all elements in a list that are greater than a given value y. lst = [3, 10, 3, 2, 5, 1, 11] y = 2 filtered_lst = [x for x in lst if x>y] print(filtered_lst) # [3, 10, 3, 5, 11] You use the condition
You can use the same idea with the less than
operator Python Filter List CountHow can you count elements under a certain condition in Python? For example, what if you want to count all even values in a list? Or all prime numbers? Or all strings that start with a certain character? There are multiple ways to accomplish this, let’s discuss them one by one. Say, you have a condition for each element For example, this condition returns def condition(x):
return x > 10
print(condition(10))
# False
print(condition(2))
# False
print(condition(11))
# True
But you can also define more complicated conditions such as checking if they are prime numbers. Python List Count IfHow can you count the elements of the list IF the condition is met? The answer is to use a simple generator expression >>> def condition(x): return x>10 >>> lst = [10, 11, 42, 1, 2, 3] >>> sum(condition(x) for x in lst) 2 The result indicates that there are two elements that are larger than 10. You used a generator expression that returns an iterator of Booleans. Note that the Boolean Python List Count Greater / Smaller ThanIf you want to determine the number of elements that are greater than or smaller than a specified value, just modify the condition in this example: >>> def condition(x): return x>10 >>> lst = [10, 11, 42, 1, 2, 3] >>> sum(condition(x) for x in lst) 2 For example, to find the number of elements smaller than 5, use the condition >>> lst = [10, 11, 42, 1, 2, 3] >>> sum(x<5 for x in lst) 3 Python List Count Zero / Non-ZeroTo count the number of zeros in a given list, use the To count the number of non-zeros in a given list, you should use conditional counting as discussed before: def condition(x):
return x!=0
lst = [10, 11, 42, 1, 2, 0, 0, 0]
print(sum(condition(x) for x in lst))
# 5
Python List Count Lambda + MapAn alternative is to use a combination of the map and the lambda function. Related articles:
Here’s the code: >>> sum(map(lambda x: x%2==0, [1, 2, 3, 4, 5])) 2 You count the number of even integers in the list.
The result is the number of elements for which the condition evaluates to Python Filter List By String LengthGiven a list of strings. How to obtain all elements that have more than coders = ['Ann', 'Alice', 'Frank', 'Pit'] filtered = [x for x in coders if len(x)>3] print(filtered) # ['Alice', 'Frank'] The list comprehension statement Python Filter List NoneHow to remove all For example, you have the list coders = ['Alice', None, 'Ann', None, None, 'Bob'] filtered = [x for x in coders if x] print(filtered) # ['Alice', 'Ann', 'Bob'] In Python, each element has an associated Boolean value so you can use any Python object as a condition. The value Python Filter JSON ListProblem: Say, you’ve got a JSON list object. You want to filter the list based on an attribute. How to accomplish that? Example: Given the following JSON list. json = [
{
"user": "alice",
"type": "free"
},
{
"user": "ann",
"type": "paid"
},
{
"user": "bob",
"type": "paid"
}
]You want to find all users that have a [
{
"user": "ann",
"type": "paid"
},
{
"user": "bob",
"type": "paid"
}
]
Solution: Use list
comprehension json = [
{
"user": "alice",
"type": "free"
},
{
"user": "ann",
"type": "paid"
},
{
"user": "bob",
"type": "paid"
}
]
filtered = [x for x in json if x['type']=='paid']
print(filtered)
# [{'user': 'ann', 'type': 'paid'},
# {'user': 'bob', 'type': 'paid'}]Only Ann and Bob have a paid account and pass the test Python Filter List One LineWant to filter your list by a given condition in one line of code? Use the list comprehension statement Here’s an example: lst = ['Alice', 3, 5, 'Bob', 10] # ONE-LINER: f = [x for x in lst if type(x)==str] print(f) # ['Alice', 'Bob'] The one-liner filters all elements in the list and checks whether they are of type string. If they are, they pass the test and are included in the new list. If you like one-liners, you’ll love my Python One-Liner book (NoStarch Press 2020). It shows you exactly how to write Pythonic code and compress your thinking and coding to the most minimalistic form. Python Filter List Efficiently: filter() vs List Comprehension[Spoiler] Which is faster to filter a list: To answer this question, I’ve written a short script that tests the runtime performance of filtering large lists of increasing sizes using the My thesis is that the list comprehension method should be slightly faster for larger list sizes because it leverages the efficient cPython implementation of list comprehension and doesn’t need to call an extra function. I used my notebook with an Intel(R) Core(TM) i7-8565U 1.8GHz processor (with Turbo Boost up to 4.6 GHz) and 8 GB of RAM. Then, I created 100 lists with both methods with sizes ranging from 10,000 elements to 1,000,000 elements. As elements, I simply incremented integer numbers by one starting from 0. Here’s the code I used to measure and plot the results: which method is faster— import time
# Compare runtime of both methods
list_sizes = [i * 10000 for i in range(100)]
filter_runtimes = []
list_comp_runtimes = []
for size in list_sizes:
lst = list(range(size))
# Get time stamps
time_0 = time.time()
list(filter(lambda x: x%2, lst))
time_1 = time.time()
[x for x in lst if x%2]
time_2 = time.time()
# Calculate runtimes
filter_runtimes.append((size, time_1 - time_0))
list_comp_runtimes.append((size, time_2 - time_1))
# Plot everything
import matplotlib.pyplot as plt
import numpy as np
f_r = np.array(filter_runtimes)
l_r = np.array(list_comp_runtimes)
print(filter_runtimes)
print(list_comp_runtimes)
plt.plot(f_r[:,0], f_r[:,1], label='filter()')
plt.plot(l_r[:,0], l_r[:,1], label='list comprehension')
plt.xlabel('list size')
plt.ylabel('runtime (seconds)')
plt.legend()
plt.savefig('filter_list_comp.jpg')
plt.show()
The code compares the runtimes of the Here’s the resulting plot that compares the runtime of the two methods. On the x axis, you can see the list size from 0 to 1,000,000 elements. On the y axis, you can see the runtime in seconds needed to execute the respective functions. The resulting plot shows that both methods are extremely fast for a few tens of thousands of elements. In fact, they are so fast that the But as you increase the size of the lists to hundreds of thousands of elements, the list comprehension method starts to win: For large
lists with one million elements, filtering lists with list comprehension is 40% faster than the built-in The reason is the efficient implementation of the list comprehension statement. An interesting observation is the following though. If you don’t convert the filter function to a list, you get the following result: Suddenly the The explanation is simple: the filter function returns an iterator, not a list. The iterator doesn’t need to compute
a single element until it is requested to compute the So, the Programming Humor – PythonWhere to Go From HereThis tutorial has shown you the ins and outs of the If you love coding and you want to do this full-time from the comfort of your own home, you’re in luck: I’ve created a free webinar that shows you how I started as a Python freelancer after my computer science studies working from home (and seeing my kids grow up) while earning a full-time income working only part-time hours. Webinar: How to Become Six-Figure Python Freelancer? Join 21,419 ambitious Python coders. It’s fun! While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students. To help students reach higher levels of Python success, he founded the programming education website Finxter.com. He’s author of the popular programming book Python One-Liners (NoStarch 2020), coauthor of the Coffee Break Python series of self-published books, computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide. His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here. How do you filter a list by condition in Python?To filter a list in Python, use the built-in filter() function.. A for loop goes through each element of a list.. It checks if an element satisfies a condition.. Based on the condition it adds the element to the result.. How do you filter the elements based on a function in a Python list?Python has a built-in function called filter() that allows you to filter a list (or a tuple) in a more beautiful way. The filter() function iterates over the elements of the list and applies the fn() function to each element. It returns an iterator for the elements where the fn() returns True .
How do you filter data from a list?Run the Advanced Filter
Select a cell in the data table. On the Data tab of the Ribbon, in the Sort & Filter group, click Advanced, to open the Advanced Filter dialog box. For Action, select Filter the list, in-place. For List range, select the data table.
How do you check if every element of a list satisfies a condition in Python?Use all() to check if every element of a list satisfies a condition. Use the syntax condition for item in list to build a generator expression that checks each item in list against a given condition .
|
