我试图抓取亚马逊的产品页面( https://www.amazon.com/dp/B0B6TR2GTJ), 代码如下:
import requests url = "https://www.amazon.com/dp/B0B6TR2GTJ" headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36', 'Accept-Language': 'en-US, en;q=0.5' } r= requests.get(url, headers = headers) print(r.status_code) print("-------------------") doc = pq(r.text) print(doc("title")) print("-------------------") print(r.text) 结果如下(被判断为机器人了): Headers 尝试了各种写法, 都是一样的结果.
503 ------------------- <title>Sorry! Something went wrong!</title> ------------------- <!-- To discuss automated access to Amazon data please contact [email protected]. For information about migrating to our APIs refer to our Marketplace APIs at https://developer.amazonservices.com/ref=rm_5_sv, or our Product Advertising API at https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html/ref=rm_5_ac for advertising use cases. --> <!doctype html> ...... 我爬虫还在初学阶段, 有没有前辈大神帮帮我. 万分感谢
5
Select Language