How does python detect typing?
Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Python Type Checking Show
In this guide, you will get a look into Python type checking. Traditionally, types have been handled by the Python interpreter in a flexible but implicit way. Recent versions of Python allow you to specify explicit type hints that can be used by different tools to help you develop your code more efficiently. In this tutorial, you’ll learn about the following:
This is a comprehensive guide that will cover a lot of ground. If you want to just get a quick glimpse of how type hints work in Python, and see whether type checking is something you would include in your code, you don’t need to read all of it. The two sections Hello Types and Pros and Cons will give you a taste of how type checking works and recommendations about when it’ll be useful. Type SystemsAll programming languages include some kind of type system that formalizes which categories of objects it can work with and how those categories are treated. For instance, a type system can define a numerical type, with Dynamic TypingPython is a dynamically typed language. This means that the Python interpreter does type checking only as code runs, and that the type of a variable is allowed to change over its lifetime. The following dummy examples demonstrate that Python has dynamic typing: >>>
In the first example, the branch Next, let’s see if variables can change type: >>>
Static TypingThe opposite of dynamic typing is static typing. Static type checks are performed without running the program. In most statically typed languages, for instance C and Java, this is done as your program is compiled. With static typing, variables generally are not allowed to change types, although mechanisms for casting a variable to a different type may exist. Let’s look at a quick example from a statically typed language. Consider the following Java snippet:
The first line declares that the variable name Python will always remain a dynamically typed language. However, PEP 484 introduced type hints, which make it possible to also do static type checking of Python code. Unlike how types work in most other statically typed languages, type hints by themselves don’t cause Python to enforce types. As the name says, type hints just suggest types. There are other tools, which you’ll see later, that perform static type checking using type hints. Duck TypingAnother term that is often used when talking about Python is duck typing. This moniker comes from the phrase “if it walks like a duck and it quacks like a duck, then it must be a duck” (or any of its variations). Duck typing is a concept related to dynamic typing, where the type or the class of an object is less important than the methods it defines. Using duck typing you do not check types at all. Instead you check for the presence of a given method or attribute. As an example, you can call >>>
Note that the call to
In order to call Duck typing is somewhat supported when doing static type checking of Python code, using structural subtyping. You’ll learn more about duck typing later. Hello TypesIn this section you’ll see how to add type hints to a function. The following function turns a text string into a headline by adding proper capitalization and a decorative line:
By default the function returns the headline left aligned with an underline. By setting the >>>
It’s time for our first type hints! To add information about types to the function, you simply annotate its arguments and return value as follows:
The In terms of style, PEP 8 recommends the following:
Adding
type hints like this has no runtime effect: they are only hints and are not enforced on their own. For instance, if we use a wrong type for the (admittedly badly named) >>>
To catch this kind of error you can use a static type checker. That is, a tool that checks the types of your code without actually running it in the traditional sense. You might already have such a type checker built into your editor. For instance PyCharm immediately gives you a warning: The most common tool for doing type checking is Mypy though. You’ll get a short introduction to Mypy in a moment, while you can learn much more about how it works later. If you don’t already have Mypy on your system, you can install it using Put the following code in a file called
This is essentially the same code you saw earlier: the definition of Now run Mypy on this code:
Based on the type hints, Mypy is able to tell us that we are using the wrong type on line 10. To fix the issue in the code you should change the value of the
Here you’ve changed
The success message confirms there were no type errors detected. Older versions of Mypy used to indicate this by showing no output at all. Furthermore, when you run the code you see the expected output:
The first headline is aligned to the left, while the second one is centered. Pros and ConsThe previous section gave you a little taste of what type checking in Python looks like. You also saw an example of one of the advantages of adding types to your code: type hints help catch certain errors. Other advantages include:
Of course, static type checking is not all peaches and cream. There are also some downsides you should consider:
You’ll later learn about the To get some idea about this, create two files: On Linux it’s
quite easy to check how much time the
So running the
Based
on this test, the import of the One of the advertised improvements in Python 3.7 is faster startup. Let’s see if the results are different:
Indeed, the general startup time is reduced by about 8 milliseconds, and the time to import Using
There are similar tools on other platforms. Python itself comes with the
While you get a result, you should be suspicious about the result: 0.1 microsecond is more than 100000 times faster than what
To get reasonable results you can tell
These results are on the same scale as the results from The conclusion in both these cases is
that importing The New In Python 3.7 there is also a new command line option that can be used to figure out how much time imports take. Using
This shows a similar result. Importing So, should you use static type checking in your own code? Well, it’s not an all-or-nothing question. Luckily, Python supports the concept of gradual typing. This means that you can gradually introduce types into your code. Code without type hints will be ignored by the static type checker. Therefore, you can start adding types to critical components, and continue as long as it adds value to you. Looking at the lists above of pros and cons you’ll notice that adding types will have no effect on your running program or the users of your program. Type checking is meant to make your life as a developer better and more convenient. A few rules of thumb on whether to add types to your project are:
In his excellent article The State of Type Hints in Python Bernát Gábor recommends that “type hints should be used whenever unit tests are worth writing.” Indeed, type hints play a similar role as tests in your code: they help you as a developer write better code. Hopefully you now have an idea about how type checking works in Python and whether it’s something you would like to employ in your own projects. In the rest of this guide, we’ll go into more detail about the Python type system, including how you run static type checkers (with particular focus on Mypy), how you type check code that uses libraries without type hints, and how you use annotations at runtime. AnnotationsAnnotations were introduced in Python 3.0, originally without any specific purpose. They were simply a way to associate arbitrary expressions to function arguments and return values. Years later, PEP 484 defined how to add type hints to your Python code, based off work that Jukka Lehtosalo had done on his Ph.D. project—Mypy. The main way to add type hints is using annotations. As type checking is becoming more and more common, this also means that annotations should mainly be reserved for type hints. The next sections explain how annotations work in the context of type hints. Function AnnotationsFor functions, you can annotate arguments and the return value. This is done as follows:
For arguments the syntax is The following simple example adds annotations to a function that calculates the circumference of a circle:
When running the code, you can also inspect the annotations. They are stored in a special >>>
Sometimes you might be confused by how Mypy is interpreting your type hints. For those cases there are special Mypy expressions:
Next, run this code through Mypy:
Even without any annotations Mypy has correctly inferred the types of the built-in If Mypy says that “Name ‘ Variable AnnotationsIn the definition of However, sometimes the type checker needs help in figuring out the types of variables as well. Variable annotations were defined in PEP 526 and introduced in Python 3.6. The syntax is the same as for function argument annotations:
The variable Annotations of variables are stored in the module level >>>
You’re allowed to annotate a variable without giving it a value. This adds the annotation to the >>>
Since no value was assigned to Playing With Python Types, Part 1Up until now you’ve only used basic types like In this section you will learn more about this type system, while implementing a simple card game. You will see how to specify:
After a short detour into some type theory you will then see even more ways to specify types in Python. You can find the code examples from this section here. Example: A Deck of CardsThe following example shows an implementation of a regular (French) deck of cards:
Each card is represented as a tuple of strings denoting the suit and rank. The deck is represented as a list of cards. Finally,
You will see how to extend this example into a more interesting game as we move along. Sequences and MappingsLet’s add type hints to our card game. In
other words, let’s annotate the functions With simple types like >>>
With composite types, you are allowed to do the same: >>>
However, this does not really tell the full story. What will be the types of Instead, you should use the special types defined in the
>>>
Note that each of these types start with a capital letter and that they all use square brackets to define item types:
The Let’s return to the card game. A card is represented by a tuple of two strings. You can write this as
In addition to the return value, you’ve also added the In many cases your functions will expect some kind of sequence, and not really care whether it is a list or a tuple. In these cases you should use
Using Type AliasesThe type hints might become quite oblique when working with nested types like the deck of cards. You may need to stare at Now consider how you would annotate
That’s just terrible! Recall that type annotations are regular Python expressions. That means that you can define your own type aliases by assigning them to new variables. You can for instance create
Using
these aliases, the annotations of
Type aliases are great for making your code and its intent clearer. At the same time, these aliases can be inspected to see what they represent: >>>
Note that when printing Functions Without Return ValuesYou may know that functions without an explicit return still return >>>
While such functions technically return something, that return value is not useful. You should add
type hints saying as much by using
The annotations help catch the kinds of subtle bugs where you are trying to use a meaningless return value. Mypy will give you a helpful warning:
Note that being explicit about a function not returning anything is different from not adding a type hint about the return value:
In this latter case Mypy has no information about the return value so it will not generate any warning:
As a more exotic case, note that you can also annotate functions that are never expected to return normally. This is done using
Since Example: Play Some CardsLet’s return to our card game example. In this second version of the game, we deal a hand of cards to each player as before. Then a start player is chosen and the players take turns playing their cards. There are not really any rules in the game though, so the players will just play random cards:
Note that in addition to changing
In this example, player The Any Type
This means more or less what it says:
While Mypy will correctly infer that
You’ll see a better way shortly. First though, let’s have a more theoretical look at the Python type system, and the special role Type TheoryThis tutorial is mainly a practical guide and we will only scratch the surface of the theory underpinning Python type hints. For more details PEP 483 is a good starting point. If you want to get back to the practical examples, feel free to skip to the next section. SubtypesOne important concept is that of subtypes. Formally, we say that a type
These two conditions guarantees that even if type For a concrete example, consider >>>
Since 0 and 1 are both integers, the first condition holds. Above you can see that Booleans can be added together, but they can also do anything else integers can. This is the second condition above. In other words, The importance of subtypes is that a subtype can always pretend to be its supertype. For instance, the following code type checks as correct:
Subtypes are somewhat related to subclasses. In fact all subclasses corresponds to subtypes, and Covariant, Contravariant, and InvariantWhat happens when you use subtypes inside composite types? For instance, is
In general, you don’t need to keep these expression straight. However, you should be aware that subtypes and composite types may not be simple and intuitive. Gradual Typing and Consistent TypesEarlier we mentioned
that Python supports gradual typing, where you can gradually add type hints to your Python code. Gradual typing is essentially made possible by the Somehow The type The type checker only complains about inconsistent types. The takeaway is therefore that you will never see type errors arising from the This means that you can use Do remember, though, if you use Playing With Python Types, Part 2Let’s return to our practical examples. Recall that you were trying to annotate the general
The problem with using
Type VariablesA type variable is a special variable that can take on any type, depending on the situation. Let’s create a type variable that will effectively encapsulate the behavior of
A type variable must be defined using
Consider a few other examples:
The first two examples should have type
As you’ve already seen Note that none of these examples raised a type error. Is there a way to tell the type checker that You can constrain type variables by listing the acceptable types:
Now
Also note
that in the second example the type is considered In our card game we want to restrict
We briefly mentioned that Duck Types and ProtocolsRecall the following example from the introduction:
The answer hides behind the academic sounding term structural subtyping. One way to categorize type systems is by whether they are nominal or structural:
There is ongoing work to bring a full-fledged structural type system to Python through PEP 544 which aims at adding a concept called protocols. Most of PEP 544 is already implemented in Mypy though. A protocol specifies one or more methods that must be implemented. For example, all classes defining
Other
examples of protocols defined in the You can also define your own protocols. This is done by inheriting from
At
the time of writing the support for self-defined protocols is still experimental and only available through the The Optional TypeA common pattern in Python is to use
In the card example, the
The
challenge this creates for type hinting is that in general In order to annotate such arguments you can use the
The Note that when using either
Mypy tells you that you have not taken care of the case where
Example: The Object(ive) of the GameLet’s rewrite the card game to be more object-oriented. This will allow us to discuss how to properly annotate classes and methods. A more or less direct translation of our card game into code that uses classes for
Now let’s add types to this code. Type Hints for MethodsFirst of all type hints for methods work much the same as type hints for functions. The only difference is that the
Note that the Classes as TypesThere is a correspondence between classes and types. For example, all instances of the For example, a
Mypy is able to connect your use of This doesn’t work as cleanly though when you need to refer to the class currently being defined. For example, the Instead, you are allowed to use string literals in annotations. These strings will only be evaluated by the type checker later, and can therefore contain self and forward references. The
Note that the
Usually annotations are not used at runtime. This has given wings to the idea of postponing the evaluation of annotations. Instead of evaluating annotations as Python expressions and storing their value, the proposal is to store the string representation of the annotation and only evaluate it when needed. Such functionality is planned to become standard in the still mythical Python 4.0. However, in Python 3.7 and later, forward references are available through a
With the Returning self or clsAs noted, you should typically not annotate the There is one case where you might want to annotate
While the code runs without problems, Mypy will flag a problem:
The
issue is that even though the inherited In cases like this you want to be more careful to make sure the annotation is correct. The return type should match the type of
There are a few things to note in this example:
Annotating *args and **kwargsIn the object oriented version of the game, we added the option to name the players on the command line. This is done by listing player names after the name of the program:
This is implemented by unpacking and passing in Regarding type annotations: even though
Similarly, if you have a function or method accepting
CallablesFunctions are first-class objects in Python. This means that you can use functions as arguments to other functions. That also means that you need to be able to add type hints representing functions. Functions, as well as lambdas, methods and classes, are represented by In the following example, the function
Note the annotation of the Most callable types can be annotated in a similar manner. However, if you need more flexibility, check out callback protocols and extended callable types. Example: HeartsLet’s end with a full example of the game of Hearts. You might already know this game from other computer simulations. Here is a quick recap of the rules:
More details can be found found online. There are not many new typing concepts in this example that you have not already seen. We’ll therefore not go through this code in detail, but leave it as an example of annotated code. You can download this code and other examples from GitHub:
Here are a few points to note in the code:
When starting the game, you control the first player. Enter numbers to choose which cards to play. The following is an example of game play, with the highlighted lines showing where the player made a choice:
Static Type CheckingSo far you have seen how to add type hints to your code. In this section you’ll learn more about how to actually perform static type checking of Python code. The Mypy ProjectMypy was started by Jukka Lehtosalo during his Ph.D. studies at Cambridge around 2012. Mypy was originally envisioned as a Python variant with seamless dynamic and static typing. See Jukka’s slides from PyCon Finland 2012 for examples of the original vision of Mypy. Most of those original ideas still play a big part in the Mypy project. In fact, the slogan “Seamless dynamic and static typing” is still prominently visible on Mypy’s home page and describes the motivation for using type hints in Python well. The biggest change since 2012 is that Mypy is no longer a variant of Python. In its first versions Mypy was a stand-alone language that was compatible with Python except for its type declarations. Following a suggestion by Guido van Rossum, Mypy was rewritten to use annotations instead. Today Mypy is a static type checker for regular Python code. Running MypyBefore running Mypy for the first time, you must install the program. This is
most easily done using With Mypy installed, you can run it as a regular command line program: Running Mypy on your There are many available options when type checking your code. As Mypy is still under very active development, command line options are liable to change between versions. You should refer to Mypy’s help to see which settings are default on your version:
Additionally, the Mypy command line documentation online has a lot of information. Let’s look at some of the most common options. First of all, if you are using third-party packages without type hints, you may want to silence Mypy’s warnings about these. This can be done with the The following example uses Numpy to calculate and print the cosine of several numbers:
Note that
The actual output of this example is not important. However, you should note
that the argument You can run Mypy on this file as usual:
These warnings may not immediately make much sense to you, but you’ll learn about stubs and typeshed soon. You can essentially read the warnings as Mypy saying that the Numpy package does not contain type hints. In most cases, missing type hints in third-party packages is not something you want to be bothered with so you can silence these messages:
If you use the Two less intrusive ways of handling third-party packages are using type comments or configuration files. In a simple example as the one above, you can silence the
The literal If you have several files, it might be easier to keep track of which
imports to ignore in a configuration file. Mypy reads a file called The following configuration file will ignore that Numpy is missing type hints:
There are many options that can be specified in the configuration file. It is also possible to specify a global configuration file. See the documentation for more information. Adding StubsType hints are available for all the packages in the Python standard library. However, if you are using third-party packages you’ve already seen that the situation can be different. The following example uses the Parse package to do simple text parsing. To follow along you should first install Parse: Parse can be used to recognize simple patterns. Here is a small program that tries its best to figure out your name:
The main flow is defined in the last three lines: ask for your name, parse the answer, and print a greeting. The The program can be used as follows:
Note that even though I answer Let’s add a small bug to the program, and see if Mypy is able to help us detect it. Change line 16 from Next run Mypy on the program:
Mypy
prints a similar error to the one you saw in the previous section: It doesn’t know about the
Unfortunately, ignoring the import means that Mypy has no way of discovering the bug in our program. A better solution would be to add type hints to the Parse package itself. As Parse is open source you can actually add types to the source code and send a pull request. Alternatively, you can add the types in a stub file. A stub file is a text file that contains the signatures of methods and functions, but not their implementations. Their main function is to add type hints to code that you for some reason can’t change. To show how this works, we will add some stubs for the Parse package. First of all, you should put all your stub files inside one common
directory, and set the
You can set the variable permanently by adding the line to your Next, create a file inside your stubs directory that you call
If you have set everything up correctly, you should see this new error message. Mypy uses the new The following example does not add types for the whole
The ellipsis Finally Mypy is able to spot the bug we introduced:
This points straight to line 16 and the fact that we return a TypeshedYou’ve seen how to use stubs to add type hints without changing the source code itself. In the previous section we added some type hints to the third-party Parse package. Now, it wouldn’t be very effective if everybody needs to create their own stubs files for all third-party packages they are using. Typeshed is a Github repository that contains type hints for the Python standard library, as well as many third-party packages. Typeshed comes included with Mypy so if you are using a package that already has type hints defined in Typeshed, the type checking will just work. You can also contribute type hints to Typeshed. Make sure to get the permission of the owner of the package first though, especially because they might be working on adding type hints into the source code itself—which is the preferred approach. Other Static Type CheckersIn this tutorial, we have mainly focused on type checking using Mypy. However, there are other static type checkers in the Python ecosystem. The PyCharm editor comes with its own type checker included. If you are using PyCharm to write your Python code, it will be automatically type checked. Facebook has developed Pyre. One of its stated goals is to be fast and performant. While there are some differences, Pyre functions mostly similar to Mypy. See the documentation if you’re interested in trying out Pyre. Furthermore, Google has created Pytype. This type checker also works mostly the same as Mypy. In addition to checking annotated code, Pytype has some support for running type checks on unannotated code and even adding annotations to code automatically. See the quickstart document for more information. Using Types at RuntimeAs a final note, it’s possible to use type hints also at runtime during execution of your Python program. Runtime type checking will probably never be natively supported in Python. However, the type hints are available at runtime in the Another use of type hints is for translating your Python code to C and compiling it for optimization. The popular Cython project uses a hybrid C/Python language to write statically typed Python code. However, since version 0.27 Cython has also supported type annotations. Recently, the Mypyc project has become available. While not yet ready for general use, it can compile some type annotated Python code to C extensions. ConclusionType hinting in Python is a very useful feature that you can happily live without. Type hints don’t make you capable of writing any code you can’t write without using type hints. Instead, using type hints makes it easier for you to reason about code, find subtle bugs, and maintain a clean architecture. In this tutorial you have learned how type hinting works in Python, and how gradual typing makes type checks in Python more flexible than in many other languages. You’ve seen some of the pros and cons of using type hints, and how they can be added to code using annotations or type comments. Finally you saw many of the different types that Python supports, as well as how to perform static type checking. There are many resources to learn more about static type checking in Python. PEP 483 and PEP 484 give a lot of background about how type checking is implemented in Python. The Mypy documentation has a great reference section detailing all the different types available. Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Python Type Checking How does Python deal with typing?Python will always remain a dynamically typed language. However, PEP 484 introduced type hints, which make it possible to also do static type checking of Python code. Unlike how types work in most other statically typed languages, type hints by themselves don't cause Python to enforce types.
How does Python identify data type?To get the type of a variable in Python, you can use the built-in type() function. In Python, everything is an object. So, when you use the type() function to print the type of the value stored in a variable to the console, it returns the class type of the object.
Does Python typing enforce type?The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc. This module provides runtime support for type hints.
Which typing is identified with Python?Python is a dynamically typed language. That means it is not necessary to declare the type of a variable when assigning a value to it. For instance, you do not need to declare the data type of object major as string when you initialise the object with a string value of 'Tom' .
|