某多多爬虫 - V2EX
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
Northxw

某多多爬虫

  •  
  •   Northxw Apr 22, 2019 9586 views
    This topic created in 2581 days ago, the information mentioned may be changed or developed.

    某多多爬虫,anti_content 参数破解参考网上教程(因侵权问题,此处不公开链接),主要是通过是商品 API 完成商品信息的抓取,分析,存储,可视化等功能。

    Github: https://github.com/Northxw/Pinduoduo 欢迎提交 issue

    Supplement 1    Apr 22, 2019
    注意:这里抓取的站点是拼多多移动端链接。
    16 replies    2024-02-06 12:33:11 +08:00
    yepinf
        1
    yepinf  
       Apr 22, 2019 via iPhone
    请教下,pdd 除了参数加密,还有什么注意点
    有闲置服务器,想稳定爬它几个月看看
    showHand043
        2
    showHand043  
       Apr 22, 2019
    没有反爬措施嘛.比如汽车之家,大众点评那种
    AnjingJingan
        3
    AnjingJingan  
       Apr 22, 2019
    pdd.txt 不存在
    Northxw
        4
    Northxw  
    OP
       Apr 22, 2019
    @AnjingJingan 跑一边程序就生成了,不可能把大文件放到 github 上啊。
    Northxw
        5
    Northxw  
    OP
       Apr 22, 2019
    @yepinf 目前还未发现其他反爬措施,不过大规模抓取的时候建议上代理。你先试试。
    Northxw
        6
    Northxw  
    OP
       Apr 22, 2019
    @showHand043 反爬措施就是参数加密啊(注意:注意看我的抓取链接),汽车之家和大众点评是网页字体反爬。
    AnjingJingan
        7
    AnjingJingan  
       Apr 22, 2019
    @Northxw 跑程序报了这个错
    Northxw
        8
    Northxw  
    OP
       Apr 22, 2019
    @AnjingJingan ???贴代码,我看看什么错误。
    Northxw
        9
    Northxw  
    OP
       Apr 22, 2019
    @AnjingJingan 或者你加我 QQ:2443498314, 密保问题:佩奇(备注来意)
    luanguang
        10
    luanguang  
       Apr 22, 2019
    原本想 star 的,看到这么多有趣的爬虫,还是 follow 吧。
    Northxw
        11
    Northxw  
    OP
       Apr 22, 2019
    @luanguang 哈哈 都行的, 互相学习
    stcasshern
        12
    stcasshern  
       Apr 22, 2019
    露珠下步计划是啥,感觉下步分析比较有趣
    Northxw
        13
    Northxw  
    OP
       Apr 22, 2019
    @stcasshern ???哈哈,你想要什么
    ricky0603
        14
    ricky0603  
       May 21, 2020
    楼主的账号问题是怎么解决的?我一个账号抓 300+条数据就被 ban
    funtanstic
        15
    funtanstic  
       Feb 5, 2024
    大佬问题解决了么?账号很容易被 ban
    Northxw
        16
    Northxw  
    OP
       Feb 6, 2024
    @funtanstic 兄弟,几年前的问题了,我现在已经不搞爬虫了,账号被 ban 就挂代理(真实代理,贵一点的,或者自己维护 ip 池)
    About     Help     Advertise     Blog     API     FAQ     Solana     1029 Online   Highest 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 37ms UTC 23:08 PVG 07:08 LAX 16:08 JFK 19:08
    Do have faith in what you're doing.
    ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86