May 30, 2021 Article blog
If you have a habit of reading source code, you may see some good code often appear with the "with" keyword statement, it is usually used in what scenario? Let's talk about with and context managers today.
For system resources such as files, database connections, sockets, one of the things an application must do after opening those resources and executing business logic is to shut down (disconnect) the resource.
For example, the Python program opens a file, writes to the file, and after writing, closes the file, otherwise what will happen? In extreme cases, an "Too many open files" error occurs because the maximum number of files that the system allows you to open is limited.
Similarly, for databases, if there are too many connections to close in time, "Can not connect to MySQL server Too many connections" may occur because database connections are a very expensive resource and cannot be created without restrictions.
Let's see how to close a file correctly.
def test1():
f = open("output.txt", "w")
F.write ("Python Zen")
f.close()
There is a potential problem with this writing, if an exception occurs during the call to write that prevents subsequent code from continuing and the close method cannot be called normally, so resources are kept occupied by the program and cannot be released. So how do you improve your code?
def test2():
f = open("output.txt", "w")
try:
F.write ("Python Zen")
except IOError:
print("oops error")
finally:
f.close()
An improved version of the program is a try capture of code where an exception is likely to occur, using a try/finally statement that indicates that if an exception occurs in the try block, subsequent code is no longer executed and jumps directly to the except block. I n any case, the code for the final block will eventually be executed. Therefore, as long as you put close in the finally code, the file will definitely close.
def test3():
with open("output.txt", "w") as f:
F.write ("Python Zen")
A more concise and elegant way is to use the with keyword. T
he return value of the open method is assigned to variable f, and when you leave the block of code with, the system automatically calls the f.close() method, which functions the same as using the try/finally statement. S
o how does it work? A
nother concept involved before talking about the principle of with is context manager.
Any object that implements the __enter__() and __exit__() methods can be called a context manager, which can use the with keyword. Obviously, the file object also implements the context manager.
So how do file objects implement both methods? We can simulate implementing a file class of our own, and let that class implement __enter__() and __exit__() methods.
class File():
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
def __enter__(self):
print("entering")
self.f = open(self.filename, self.mode)
return self.f
def __exit__(self, *args):
print("will exit")
self.f.close()
__enter__() method returns the resource object, here is the file object that you are about to open, __exit__ () method handles some cleanup work.
Because the File class implements the context manager, you can now use the with statement.
with File('out.txt', 'w') as f:
print("writing")
f.write('hello, python')
This way, you don't have to explicitly call the close method, which is called automatically by the system, even if an exception is encountered in the middle.
Python also provides a contextmanager decorator that further simplifies the implementation of the context manager. T he function is split into two parts by yield, the statement before yield is executed in the __enter__ method, and the statement after yield is executed in the __exit__ method. T he value immediately after yield is the return value of the function.
from contextlib import contextmanager
@contextmanager
def my_open(path, mode):
f = open(path, mode)
yield f
f.close()
invoke:
with my_open('out.txt', 'w') as f:
f.write("hello , the simplest context manager")
Python provides a follow-up cleanup of the with syntax to simplify resource operations, an alternative to try/finally, based on a context manager.
In addition, Python provides a contextmanager decorator that further simplifies the implementation of the up-and-down manager.
Recommended reading: Python3 Getting Started, Python3 Advanced