Can python 2 scripts run in python 3?
Calling different python versions from each other can be done very elegantly using execnet. The following function does the charm: Show
Example: A
Then the following function calls
result in
What happened is that a 'gateway' was instantiated waiting for an argument list with Brett Cannon Abstract With Python 3 being the future of Python while Python 2 is still in active use, it is good to have your project available for both major releases of Python. This guide is meant to help you figure out how best to support both Python 2 & 3 simultaneously. If you are looking to port an extension module instead of pure Python code, please see Porting Extension Modules to Python 3. If you would like to read one core Python developer’s take on why Python 3 came into existence, you can read Nick Coghlan’s Python 3 Q & A or Brett Cannon’s Why Python 3 exists. For help with porting, you can view the archived python-porting mailing list. The Short Explanation¶To make your project be single-source Python 2/3 compatible, the basic steps are:
Note Note: Using Details¶A key point about supporting Python 2 & 3 simultaneously is that you can start today! Even if your dependencies are not supporting Python 3 yet that does not mean you can’t modernize your code now to support Python 3. Most changes required to support Python 3 lead to cleaner code using newer practices even in Python 2 code. Another key point is that modernizing your Python 2 code to also support Python 3 is largely automated for you. While you might have to make some API decisions thanks to Python 3 clarifying text data versus binary data, the lower-level work is now mostly done for you and thus can at least benefit from the automated changes immediately. Keep those key points in mind while you read on about the details of porting your code to support Python 2 & 3 simultaneously. Drop support for Python 2.6 and older¶While you can make Python 2.5 work with Python 3, it is much easier if you only have to work with Python 2.7. If dropping Python 2.5 is not an option then the six project can help
you support Python 2.5 & 3 simultaneously ( If you are able to skip Python 2.5 and older, then the required changes to your code should continue to look and feel like idiomatic Python code. At worst you will have to use a function instead of a method in some instances or have to import a function instead of using a built-in one, but otherwise the overall transformation should not feel foreign to you. But you should aim for only supporting Python 2.7. Python 2.6 is no longer freely supported and thus is not receiving bugfixes. This means you will have to work around any issues you come across with Python 2.6. There are also some tools mentioned in this HOWTO which do not support Python 2.6 (e.g., Pylint), and this will become more commonplace as time goes on. It will simply be easier for you if you only support the versions of Python that you have to support. Make sure you specify the proper version support in your setup.py file¶In your Have good test coverage¶Once you have your code supporting the oldest version of Python 2 you want it to, you will want to make sure your test suite has good coverage. A good rule of thumb is that if you want to be confident enough in your test suite that any failures that appear after having tools rewrite your code are actual bugs in the tools and not in your code. If you want a number to aim for, try to get over 80% coverage (and don’t feel bad if you find it hard to get better than 90% coverage). If you don’t already have a tool to measure test coverage then coverage.py is recommended. Learn the differences between Python 2 & 3¶Once you have your code well-tested you are ready to begin porting your code to Python 3! But to fully understand how your code is going to change and what you want to look out for while you code, you will want to learn what changes Python 3 makes in terms of Python 2. Typically the two best ways of doing that is reading the “What’s New” doc for each release of Python 3 and the Porting to Python 3 book (which is free online). There is also a handy cheat sheet from the Python-Future project. Update your code¶Once you feel like you know what is different in Python 3 compared to Python 2, it’s time to update your code! You have a choice between two tools in porting your code automatically: Futurize and
Modernize. Which tool you choose will depend on how much like Python 3 you want your code to be. Futurize does its best to make Python 3 idioms and practices exist in Python 2, e.g. backporting the Regardless of which tool you choose, they will update your code to run under Python 3 while staying compatible with the version of Python 2 you started with. Depending on how conservative you want to be, you may want to run the tool over your test suite first and visually inspect the diff to make sure the transformation is accurate. After you have transformed your test suite and verified that all the tests still pass as expected, then you can transform your application code knowing that any tests which fail is a translation failure. Unfortunately
the tools can’t automate everything to make your code work under Python 3 and so there are a handful of things you will need to update manually to get full Python 3 support (which of these steps are necessary vary between the tools). Read the documentation for the tool you choose to use to see what it fixes by default and what it can do optionally to know what will (not) be fixed for you and what you may have to fix on your own (e.g. using Division¶In Python 3,
The reason that Text versus binary data¶In Python 2 you could use the To make the distinction between text and binary data clearer and more pronounced, Python 3 did what most languages created in the age of the internet have done and made text and binary data distinct types that cannot blindly be mixed together (Python predates widespread access to the internet). For any code that deals only with text or only binary data, this separation doesn’t pose an issue. But for code that has to deal with both, it does mean you might have to now care about when you are using text compared to binary data, which is why this cannot be entirely automated. To start, you will need to decide which APIs take text and which take binary (it is highly recommended you don’t design APIs that can take both due to the difficulty of
keeping the code working; as stated earlier it is difficult to do well). In Python 2 this means making sure the APIs that take text can work with
Making the distinction easier to handle can be accomplished by encoding and decoding between binary data and text at the edge of your code. This means that when you receive text in binary data, you should immediately decode it. And if your code needs to send text as binary data then encode it as late as possible. This allows your code to work with only text internally and thus eliminates having to keep track of what type of data you are working with. The next
issue is making sure you know whether the string literals in your code represent text or binary data. You should add a As part of this dichotomy you also need to be careful about opening files. Unless you have been working on Windows, there is a chance you have not always bothered to add the The
constructors of both Finally, the indexing of binary data requires careful handling (slicing does not require any special handling). In Python 2, To summarize:
Use feature detection instead of version detection¶Inevitably you will have code that has to choose what to do based on what version of Python is running. The best way to do this is with feature detection of whether the version of Python you’re running under supports what you need. If for some reason that doesn’t work then you should make the version check be against Python 2 and not Python 3. To help explain this, let’s look at an example. Let’s pretend that you need access to a feature of import sys if sys.version_info[0] == 3: from importlib import abc else: from importlib2 import abc The problem with this code is what happens when Python 4 comes out? It would be better to treat Python 2 as the exceptional case instead of Python 3 and assume that future Python versions will be more compatible with Python 3 than Python 2: import sys if sys.version_info[0] > 2: from importlib import abc else: from importlib2 import abc The best solution, though, is to do no version detection at all and instead rely on feature detection. That avoids any potential issues of getting the version detection wrong and helps keep you future-compatible: try: from importlib import abc except ImportError: from importlib2 import abc Prevent compatibility regressions¶Once you have fully translated your code to be compatible with Python 3, you will want to make sure your code doesn’t regress and stop working under Python 3. This is especially true if you have a dependency which is blocking you from actually running under Python 3 at the moment. To help with staying compatible, any new modules you create should have at least the following block of code at the top of it: from __future__ import absolute_import from __future__ import division from __future__ import print_function You can also run Python 2 with the You can also use the Pylint project and its Check which dependencies block your transition¶After you have made your code compatible with Python 3 you should begin to care about whether your dependencies have also been ported. The caniusepython3 project was created to help you determine which projects – directly or indirectly – are blocking you from supporting Python 3. There is both a command-line tool as well as a web interface at https://caniusepython3.com. The project also provides code which you can integrate into your test suite so that you will have a failing test when you no longer have dependencies blocking you from using Python 3. This allows you to avoid having to manually check your dependencies and to be notified quickly when you can start running on Python 3. Update your setup.py file to denote Python 3 compatibility¶Once your code works under Python 3, you should update the classifiers in your Use continuous integration to stay compatible¶Once you are able to fully run under Python 3 you will want to make sure your code always works under both Python 2 & 3. Probably the best tool for running your tests under multiple Python interpreters is tox. You can then integrate tox with your continuous integration system so that you never accidentally break Python 2 or 3 support. You may also want to use the And that’s mostly it! At this point your code base is compatible with both Python 2 and 3 simultaneously. Your testing will also be set up so that you don’t accidentally break Python 2 or 3 compatibility regardless of which version you typically run your tests under while developing. Consider using optional static type checking¶Another way to help port your code is to use a static type checker like mypy or pytype on your code. These tools can be used to analyze your code as if it’s being run under Python 2, then you can run the tool a second time as if your code is running under Python 3. By running a static type checker twice like this you can discover if you’re e.g. misusing binary data type in one version of Python compared to another. If you add optional type hints to your code you can also explicitly state whether your APIs use textual or binary data, helping to make sure everything functions as expected in both versions of Python. |