请教一下大神们,日志是怎么处理的? - V2EX
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
Sign Up Now
For Existing Member  Sign In
请不要在回答技术问题时复制粘贴 AI 生成的内容
deweixu

请教一下大神们,日志是怎么处理的?

  •  
  •   deweixu
    deweixu Dec 8, 2017 3616 views
    This topic created in 3068 days ago, the information mentioned may be changed or developed.

    我们之前对日志的处理是,每天会有一个定时任务处理头一天的日志,会选择一个关键的日志入库,然后可以下载,导出成 excel 之类的,但是随着日志数据量的巨增,发现入库不是一个好的选择,大神们是怎么弄的呢?

    8 replies    2017-12-09 09:23:28 +08:00
    duskx
        1
    duskx  
       Dec 8, 2017 via iPhone
    ELK
    cls1991
        2
    cls1991  
       Dec 8, 2017
    elk
    deweixu
        3
    deweixu  
    OP
       Dec 8, 2017
    @957204459 Elasticsearch 是不是还要把从 logstash 得到的数据存储起来?
    zen9073
        4
    zen9073  
       Dec 8, 2017
    @deweixu
    logstash 收集 log,发送到 elasticsearch 存储,然后通过 kibana 处理展示,
    roricon
        5
    roricon  
       Dec 8, 2017
    在 logstash parse log file 的时候加 filter 可以过滤掉你不关心的 log entries
    iugo
        6
    iugo  
       Dec 8, 2017
    之前想过用 Kafka + 自定义处理, 但前一段将日志移入某云. 然后只针对严重的错误进行提醒.
    qqmishi
        7
    qqmishi  
       Dec 9, 2017
    或者试试 loggly 这种日志服务
    MoliOOO
        8
    MoliOOO  
       Dec 9, 2017
    试试 Graylog
    About     Help     Advertise     Blog     API     FAQ     Solana     2401 Online   Highest 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 52ms UTC 01:50 PVG 09:50 LAX 18:50 JFK 21:50
    Do have faith in what you're doing.
    ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86