May 31, 2021 Article blog
The main points of knowledge covered in this article are as follows:
Environment: Python3.6 and Pycharm
Library: requests
The code in this article has been detailed and can be run directly.
First, the small partners in front of the screen need to install the requests library first, and the Python environment needs to be installed before installing, and if not, the editor-in-chief is here to provide the small partners with the latest Python compiler installation tutorial: Python's latest 3.9.0 compiler installation tutorial.
Once the Python environment is installed, the windows user opens the cmd command and enters the following command (the rest of the system is installed roughly the same).
pip install requests
Linux users:
sudo pip install requests
Next is the example explanation, little partners more hands-on drills now!
1, climb Baidu home page, and get page information
# Crawl Baidu page
Import Requests # Import Requests Reptile Library
Resp = Requests.get ('http://www.baidu.com') # Generates a Response object
Resp.Encoding = 'UTF-8' # Set the encoded format to UTF-8
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
2, requests library get method instance
Before you introduce a URL: httpbin.org, this site can test the HTTP request and response of various information, such as cookies, ip, headers and login verification, and support GET, POST and other methods, web development and testing is very helpful. It was written in Python and Flask and is an open source project.
Official website: http://httpbin.org/
Open source address: https://github.com/Runscope/httpbin
# get method instance
Import Requests # Import Requests Reptile Library
RESP5, climbed webpage pictures, and saved to the local.5, climb the web image and save it locally.= Requests.get ("http://httpbin.org/get") #GET method
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
3, requests library post method instance
# Post method instance
Import Requests # Import Requests Reptile Library
Resp = Requests.Post ("http://httpbin.org/post") #post method
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
4, requests library put method instance
# pT method instance
Import Requests # Import Requests Reptile Library
Resp = Requests.put ("http://httpbin.org/put") # PUT method
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
5, requests library get method ginseng
To pass parameters using the get method, there are two ways to do this:
# Get Method Method Example 1
Import Requests # Import Requests Reptile Library
Resp = Requests.get ("http://httpbin.org/get?name=w3cschool&age=100") # get
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
# Get Transfer Method Example 2
Import Requests # Import Requests Reptile Library
data = {
"name":"w3cschool",
"age":100
} # Use dictionary storage pass parameters
Resp = Requests.get ("http://httpbin.org/get", params = data) # get
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
6, requests library post method transpose
Method two that passes parameters using the post method is similar to method two that passes parameters using the get method. Here's an example:
# POST Method Example
Import Requests # Import Requests Reptile Library
data = {
"name":"w3cschool",
"age":100
} # Use dictionary storage pass parameters
Resp = Requests.Post ("http://httpbin.org/post", params = data) # pos
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 输 的
7, how to bypass the major websites anti-reptile measures, to cat-eye box office as an example:
Import Requests # Import Requests Reptile Library
URL = 'http://piaofang.maoyan.com/dashboard' # 眼 房 网网
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'
} # Set head information, camouflage browser
resp = requests.get(url, headers=headers) #
Print (resp.status_code) # Print State Code
Print (Resp.Text) # 网 信息 信息 信息
8, climb the page picture, and save to the local.
First in the E disk to establish a reptile directory, before the information can be saved, small partners can choose to save the directory, in the code to change the corresponding directory code.
Import Requests # Import Requests Reptile Library
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'
} # Set head information, camouflage browser
Resp = Requests.get ('http://7n.w3cschool.cn/statics/img/logo/[email protected]', headers = headers) #GET method to image response
FILE = Open ("E: \\ Reptile \\ Test.png", "WB") # Open a file, WB means that open a file in binary format is only used for writing
File.write (Resp.content) # Writing
File.close () # Close the file operation
Learn to use, hope that the small partners in front of the screen can be more contact, combined with the actual more operation. Recommended reading: Python static reptiles, Python Scrapy web crawlers.