python3 urllib 设置 Host header 无效问题

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

For Existing Member Sign In

This topic created in 1681 days ago, the information mentioned may be changed or developed.

import urllib.request

headers = { "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36", "Host": "" } debug = True import traceback def g_post2(url, data, domain): try: headers['Host'] = domain opener = urllib.request.build_opener() opener.addheaders = [ ('User-Agent', headers['User-Agent']), ('Host', domain) ] print(opener.addheaders) return opener.open(url, data, timeout=2).read().decode() except Exception as e: print(e) if debug: traceback.print_exc() print(g_post2('http://httpbin.org/post','data'.encode(), 'qq.com'))

输出的结果看设置的 UA 是生效了，但设置的 Host 没有生效。是不是设置的 Host 被后面的 open 覆盖掉了，但是 open 又没提供 headers 设置参数。

[('User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36'), ('Host', 'qq.com')] { "args": {}, "data": "", "files": {}, "form": { "data": "" }, "headers": { "Accept-Encoding": "identity", "Content-Length": "4", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36", "X-Amzn-Trace-Id": "Root=1-6146527f-471edcb1073f7ebe3e0672a3" }, "json": null, "origin": "123.203.104.151", "url": "http://httpbin.org/post" }

8 replies 2021-09-19 18:20:32 +08:00

Trim21

Sep 19, 2021

设置的 header 在 open 里面被覆盖掉了。

直接用 urlopen 和 Request 不就好了

with urllib.request.urlopen(urllib.request.Request(url, data, method='GET', headers=headers)) as res:
....return res.read().decode()