最近在处理一批日志,然后 macbook 下搭建了个 es,发现日志量很大的情况下启动就会爆“Too many open files in system”... - V2EX
Distributions
Ubuntu
Fedora
CentOS
中文资源站
网易开源镜像站
Braid
V2EX    Linux

最近在处理一批日志,然后 macbook 下搭建了个 es,发现日志量很大的情况下启动就会爆“Too many open files in system”...

  •  
  •   Braid Mar 28, 2017 4321 views
    This topic created in 3332 days ago, the information mentioned may be changed or developed.
    最近在处理一批日志,然后 macbook 下搭建了个 es ,发现日志量很大的情况下启动就会爆“ Too many open files in system ”,网上查了很多方法都没得到解决。我的是 2015 款的 macbookpro ( 8G 内存+ 256GB ),目前我尝试的方法是 ulimit -Sn 10240

    求除了换电脑外的解决方法。。。
    13 replies    2017-03-29 14:19:40 +08:00
    Braid
        1
    Braid  
    OP
       Mar 28, 2017
    You can increase the limits (at your own risk) with:

    sysctl -w kern.maxfiles=20480 (or whatever number you choose)
    sysctl -w kern.maxfilesperproc=18000 (or whatever number you choose)

    这个貌似可以临时解决。
    royantar
        2
    royantar  
       Mar 28, 2017
    10240 还是小吧, es 官方稳定建议 65536 或更高。

    https://www.elastic.co/guide/en/elasticsearch/reference/5.2/file-descriptors.html
    leopku
        3
    leopku  
       Mar 28, 2017
    在 vagrant 或 docker 里跑 es ,然后把 vagrant 或 es 的 limit 调到最大
    kmahyyg
        4
    kmahyyg  
       Mar 28, 2017 via Android
    ulimit -n 51200 这是最低
    janxin
        5
    janxin  
       Mar 28, 2017
    你换个电脑也是这问题,这个是系统的最大打开句柄上限默认设置太低,改大点就好了,方法上面都提供过了。

    Linux 服务器高并发下如果不改配置,一样也会遇到这问题 23333
    wweir
        6
    wweir  
       Mar 29, 2017 via Android
    直接上 1048576 这个上限
    Braid
        7
    Braid  
    OP
       Mar 29, 2017
    @kmahyyg macbook 我那个电脑 ulimit -n 10240 就是极限了。。。之后就会报错
    KIDJourney
        8
    KIDJourney  
       Mar 29, 2017
    @Braid sudo
    eimsteim
        9
    eimsteim  
       Mar 29, 2017
    弱弱问一句, es 是什么?
    Braid
        10
    Braid  
    OP
       Mar 29, 2017
    @KIDJourney 试过 sudo 了
    Braid
        11
    Braid  
    OP
       Mar 29, 2017
    @wweir 这可以,我保险设置到了:

    sysctl -w kern.maxfiles=81920
    sysctl -w kern.maxfilesperproc=40960
    Immortal
        12
    Immortal  
       Mar 29, 2017
    这个 linux 下不就是
    ulimit -n 65535
    zhyt1985
        13
    zhyt1985  
       Mar 29, 2017
    @eimsteim ElasticSearch
    About     Help     Advertise     Blog     API     FAQ     Solana     3349 Online   Highest 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 74ms UTC 13:54 PVG 21:54 LAX 06:54 JFK 09:54
    Do have faith in what you're doing.
    ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86