推荐学习书目 Learn Python the Hard Way Python Sites PyPI - Python Package Index http://diveintopython.org/toc/index.html Pocoo 值得关注的项目 PyPy Celery Jinja2 Read the Docs gevent pyenv virtualenv Stackless Python Beautiful Soup 结巴中文分词 Green Unicorn Sentry Shovel Pyflakes pytest Python 编程 pep8 Checker Styles PEP 8 Google Python Style Guide Code Style from The Hitchhiker's Guide
这是一个创建于 2457 天前的主题，其中的信息可能已经有所发展或是发生改变。
之前写的拉钩网，我今天试了一下出现下面这个
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"60.30.54.82","state":2402}
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}
网址： https://www.lagou.com/gongsi/0-1-0-0
后来我把 cookie 换了 还是这样 这是为啥呢，ip 是动态代理？ 求人指教
from ip_pool import get_ip
import requests
headers={"Cookie":'_ga=GA1.2.174518896.1523111183; user_trace_token=20180407222623-a5c90692-3a6f-11e8-b740-5254005c3644; LGUID=20180407222623-a5c90b3f-3a6f-11e8-b740-5254005c3644; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%2C%22%24device_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%7D; index_location_city=%E5%8C%97%E4%BA%AC; JSESSIOnID=ABAAABAAAGFABEF2514709505FB85F0FC824310BC7C43F2; _gid=GA1.2.1492847185.1548121054; Hm_lvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1546789367; TG-TRACK-CODE=index_search; SEARCH_ID=8cc1b952a94a496892284ac7a525daea; _gat=1; LGSID=20190123153732-bedacc90-1ee1-11e9-9486-525400f775ce; PRE_UTM=; PRE_HOST=; PRE_SITE=; PRE_LAND=https%3A%2F%2Fwww.lagou.com%2Fgongsi%2F0-1-0-0; LG_LOGIN_USER_ID=d809bbbe54ac48bf0a9ce5888befc8dbdd72485efb1d041a; _putrc=528CDA7A1053B994; login=true; unick=%E5%B2%B3%E5%BA%B7; showExpriedIndex=1; showExpriedCompanyHome=1; showExpriedMyPublish=1; hasDeliver=138; gate_login_token=b729a3ea436639fccaac9cdae984ae92c4562ed3d14bb148; LGRID=20190123153826-ded128bf-1ee1-11e9-b748-5254005c3644; Hm_lpvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1548229075',"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","Referer": "https://www.lagou.com/gongsi/0-1-0-0",}
form_data={'first': 'false', 'pn': '3', 'sortField': '0', 'havemark': '0'}
res = requests.get('https://www.lagou.com/gongsi/0-1-0-0.json', headers=headers,data=form_data,proxies=get_ip())
print(res.text)
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"60.30.54.82","state":2402}
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}
网址： https://www.lagou.com/gongsi/0-1-0-0
后来我把 cookie 换了 还是这样 这是为啥呢，ip 是动态代理？ 求人指教
from ip_pool import get_ip
import requests
headers={"Cookie":'_ga=GA1.2.174518896.1523111183; user_trace_token=20180407222623-a5c90692-3a6f-11e8-b740-5254005c3644; LGUID=20180407222623-a5c90b3f-3a6f-11e8-b740-5254005c3644; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%2C%22%24device_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%7D; index_location_city=%E5%8C%97%E4%BA%AC; JSESSIOnID=ABAAABAAAGFABEF2514709505FB85F0FC824310BC7C43F2; _gid=GA1.2.1492847185.1548121054; Hm_lvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1546789367; TG-TRACK-CODE=index_search; SEARCH_ID=8cc1b952a94a496892284ac7a525daea; _gat=1; LGSID=20190123153732-bedacc90-1ee1-11e9-9486-525400f775ce; PRE_UTM=; PRE_HOST=; PRE_SITE=; PRE_LAND=https%3A%2F%2Fwww.lagou.com%2Fgongsi%2F0-1-0-0; LG_LOGIN_USER_ID=d809bbbe54ac48bf0a9ce5888befc8dbdd72485efb1d041a; _putrc=528CDA7A1053B994; login=true; unick=%E5%B2%B3%E5%BA%B7; showExpriedIndex=1; showExpriedCompanyHome=1; showExpriedMyPublish=1; hasDeliver=138; gate_login_token=b729a3ea436639fccaac9cdae984ae92c4562ed3d14bb148; LGRID=20190123153826-ded128bf-1ee1-11e9-b748-5254005c3644; Hm_lpvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1548229075',"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","Referer": "https://www.lagou.com/gongsi/0-1-0-0",}
form_data={'first': 'false', 'pn': '3', 'sortField': '0', 'havemark': '0'}
res = requests.get('https://www.lagou.com/gongsi/0-1-0-0.json', headers=headers,data=form_data,proxies=get_ip())
print(res.text)
 | 1 baicheng10 2019-01-23 17:49:10 +08:00  4 不是想抖机灵，但我确实第一眼看成了“您操的太频繁,请稍后再访问”= =|||  |
 | 2 loveCoding 2019-01-23 17:51:14 +08:00 拉勾用的人还多吗? |
 | 3 mytsing520 PRO 禁了 token 吧 |
 | 4 supervipcard 2019-01-23 18:31:05 +08:00 先请求 https://www.lagou.com/gongsi/获取 cookie |
 | 5 zhangslob669 2019-01-23 19:08:04 +08:00 是的，现在需要先访问两个请求获取 cookies |
 | 6 nichijou 2019-01-23 19:15:32 +08:00 我就没你这么机智懂得过来问一下 http://nichijou.city/post/431M |
 | 7 Ewig OP @zhangslob669 哪两个 |
| 8 Ewig OP @supervipcard 我还要爬那个详情页公司，下一次 cookie 好像是变化的？你说获取后就可以了？ |
| 9 Ewig OP @zhangslob669 你的模拟登陆 challence 参数如何破解的 |
| 10 Ewig OP @zhangslob669 你是模拟登陆的吗？ |
| 11 supervipcard 2019-01-24 09:08:07 +08:00 @Ewig 你自己用 fiddler 抓包找一下有 Set_Cookie 的，还要请求一个 |
| 12 Ewig OP 根据你们的意思 得先模拟登陆 才可以啊，但是模拟登陆杨浦个 chaallenge 参数 |
 | 13 daimazha 2019-01-24 09:31:34 +08:00 之前都是随便请求接口就行的 |
 | 15 c4f36e5766583218 2019-01-24 10:16:31 +08:00 ip_pool 用的是啥，交流一下 |
 | 16 luofan004 2019-01-24 14:03:11 +08:00 你在这儿问，拉勾的人看不到么，滑稽.jpg |
 | 17 fakefish 2019-01-24 15:21:40 +08:00 楼主这个有解决方法了吗？我也遇到同样问题， 发现 copy cookie 过来并不行啊 |
| 18 Ewig OP @c4f36e5766583218 一个动态代理啊 |
| 20 Ewig OP @c4f36e5766583218 你现在有时间交流吗 |
| 22 fakefish 2019-01-25 16:56:15 +08:00 @Ewig 并没有， 但是只要 cookie，header 一样， 就没问题，但是问题是他的 cookie 是动态设置的， 包括 search_id 用的也和第一次 response 里 set-cookie 的值不一样 |
| 23 c4f36e5766583218 2019-01-26 13:06:21 +08:00 @Ewig 把 ip_pool 资料 /官网贴一下 |
| 24 Ewig OP @c4f36e5766583218 就是一个动态代理啊，我花钱买的啊，这个就不停的换 ip 的一个 module 啊，和这个没关系吧 |
| 25 Ewig OP 没有高手解决吗 |
 | 26 my1103 2019-04-29 17:23:28 +08:00 因为你访问的根不是真正的 url，真正的用 AJAX 提交的 |
Select Language