mysql 的 sql 优化,佬们有啥建议,被监控出慢查询 - V2EX
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
MySQL 5.5 Community Server
MySQL 5.6 Community Server
Percona Configuration Wizard
XtraBackup 搭建主从复制
Great Sites on MySQL
Percona
MySQL Performance Blog
Severalnines
推荐管理工具
Sequel Pro
phpMyAdmin
推荐书目
MySQL Cookbook
MySQL 相关项目
MariaDB
Drizzle
参考文档
http://mysql-python.sourceforge.net/MySQLdb.html
nice2cu
V2EX    MySQL

mysql 的 sql 优化,佬们有啥建议,被监控出慢查询

  •  
  •   nice2cu 2023-08-04 14:16:59 +08:00 2602 次点击
    这是一个创建于 799 天前的主题,其中的信息可能已经有所发展或是发生改变。
    SELECT a.status, a.contract_id, a.product_id, b.num AS total , SUM(a.product_num - a.contract_outstock_num) AS sumNum FROM tableA a, tableB b
    WHERE a.corpid = ? AND a.del = ? AND b.corpid = ? AND b.del = ? AND a.product_id = b.id AND a.is_produce = ? AND a.type = ?
    GROUP BY a.contract_id, a.product_id
    HAVING sumNum > total
    ORDER BY a.id, a.product_id, a.contract_id
    24 条回复    2023-08-08 11:28:43 +08:00
    nice2cu
        1
    nice2cu  
    OP
       2023-08-04 14:18:10 +08:00
    索引该建的都建了,但由于数据量比较大,还是慢,有啥建议吗
    zoharSoul
        2
    zoharSoul  
       2023-08-04 14:18:35 +08:00   1
    申请个从库在上面跑
    你这慢就对了
    nice2cu
        3
    nice2cu  
    OP
       2023-08-04 14:21:15 +08:00
    @zoharSoul 这个如果拆成俩 sql ,在内存去计算过滤,感觉也不太好,如果其中 sql 查出的数据量比较大的话
    zoharSoul
        4
    zoharSoul  
       2023-08-04 14:32:08 +08:00
    @nice2cu #3 所以就别拆.
    申请个从库在上面跑好了
    T0m008
        5
    T0m008  
       2023-08-04 14:36:22 +08:00
    可以建临时表的话,就先建一个临时表,把这个 sql 拆开,会快很多
    jenlors
        6
    jenlors  
       2023-08-04 14:54:21 +08:00
    explain 贴一下,另外你这没用 join 吧
    ljsh093
        7
    ljsh093  
       2023-08-04 16:18:01 +08:00
    用 join ,再看下能不能把 having 去了
    zjsxwc
        8
    zjsxwc  
       2023-08-04 16:25:01 +08:00
    拆 2 个 sql 不行吗?

    1. select b.id from B where b.corpid = ? AND b.del = ?
    得到 b 的 id 集合 bIdList

    2. select a.x,a.y..,SUM() as sumNum from a where a.corpid = ? AND a.del = ? AND a.product_id IN (bIdList) GROUP BY a.contract_id, a.product_id
    HAVING sumNum > total
    ORDER BY a.id, a.product_id, a.contract_id
    zjsxwc
        9
    zjsxwc  
       2023-08-04 16:27:26 +08:00
    @zjsxwc #6 合成一个 sql

    select a.x,a.y..,SUM() as sumNum from a where a.corpid = ? AND a.del = ? AND a.product_id IN (

    select b.id from B where b.corpid = ? AND b.del = ?

    ) GROUP BY a.contract_id, a.product_id
    HAVING sumNum > total
    ORDER BY a.id, a.product_id, a.contract_id
    wander555
        10
    wander555  
       2023-08-04 16:28:21 +08:00
    group by 的原因,去重最好先查出来再去
    vovoim
        11
    vovoim  
       2023-08-04 16:30:30 +08:00
    上个 vespa engine ,在 vespa engine 上做统计查询
    xudaxian520bsz
        12
    xudaxian520bsz  
       2023-08-04 16:50:29 +08:00   1
    SELECT a.status, a.contract_id, a.product_id, b.num AS total, SUM(a.product_num - a.contract_outstock_num) AS sumNum
    FROM tableA a
    INNER JOIN tableB b ON a.product_id = b.id
    WHERE a.corpid = ? AND a.del = ? AND b.corpid = ? AND b.del = ? AND a.is_produce = ? AND a.type = ?
    GROUP BY a.contract_id, a.product_id
    HAVING EXISTS(SELECT 1 FROM tableB WHERE id = a.product_id AND num < a.product_num - a.contract_outstock_num)
    ORDER BY a.id, a.product_id, a.contract_id
    nice2cu
        13
    nice2cu  
    OP
       2023-08-04 17:15:26 +08:00
    @zjsxwc total 没取到
    zjsxwc
        14
    zjsxwc  
       2023-08-04 17:23:21 +08:00   1
    @nice2cu 确实我漏了 total ,

    还是拆 2 个 sql ,但需要加一步客户端程序自己过滤。

    1. select b.id,b.num as total from B where b.corpid = ? AND b.del = ?
    得到 b 的 id 集合 bIdList 与 b.id 对应的 total 数据 map 比如 totalByBId = {
    <bId> : total
    }

    2. select a.x,a.y..,SUM() as sumNum from a where a.corpid = ? AND a.del = ? AND a.product_id IN (bIdList) GROUP BY a.contract_id, a.product_id
    ORDER BY a.id, a.product_id, a.contract_id

    对 2 的结果通过 totalByBId 过滤出满足的结果。
    nice2cu
        15
    nice2cu  
    OP
       2023-08-04 17:26:56 +08:00
    @xudaxian520bsz num < a.product_num - a.contract_outstock_num 这个条件 sumNum > total 是不是意思不一样了
    nice2cu
        16
    nice2cu  
    OP
       2023-08-04 17:30:39 +08:00
    @zjsxwc 可能第一个查询的数据会很多 加到内存算感觉会是个隐患
    xuanbg
        17
    xuanbg  
       2023-08-05 08:59:56 +08:00   1
    数据量大是没办法优化的。。。
    wengyanbin
        18
    wengyanbin  
       2023-08-05 11:33:19 +08:00
    select t.*,b.num from (select a.status, a.contract_id, a.product_id, sum() from table a where a.corpid = ? AND a.del = ? group by a.contract_id, a.product_id) as t,table b where t.product_id=b.id and t.sumNum>b.num
    zjsxwc
        19
    zjsxwc  
       2023-08-05 12:39:43 +08:00 via Android   1
    @nice2cu 不知道用 left join 会不会好一点,起码不会和你最开始直接 from 两个表搞笛卡尔积的性能那么差。



    SELECT a.status, a.contract_id, a.product_id, total , SUM(a.product_num - a.contract_outstock_num) AS sumNum

    FROM tableA a
    LEFT JOIN ( select B.id as bid, B.num as total from B where B.corpid = ? AND B.del = ? ) tb
    ON tb.bid=a.product_id

    WHERE a.corpid = ? AND a.del = ? AND a.is_produce = ? AND a.type = ?

    GROUP BY a.contract_id, a.product_id
    HAVING sumNum > total
    ORDER BYa.id, a.product_id, a.contract_id
    xchaoinfo
        20
    xchaoinfo  
       2023-08-06 15:56:49 +08:00 via Android   1
    同意 @zjsxwc 的思路,改成子查询 然后 id 去重,速度应该提高很多
    nice2cu
        21
    nice2cu  
    OP
       2023-08-07 09:23:26 +08:00
    @zjsxwc 好的 多谢大佬 我试试
    nice2cu
        22
    nice2cu  
    OP
       2023-08-07 16:05:49 +08:00
    @zjsxwc 生产试了下 从 1.7s 优化到了 0.2s 感谢大佬
    nice2cu
        23
    nice2cu  
    OP
       2023-08-07 16:42:44 +08:00
    俩表数据量分别是 2000w
    4000w
    nice2cu
        24
    nice2cu  
    OP
       2023-08-08 11:28:43 +08:00
    @zjsxwc 佬,再问个问题 现在我们生产开启了查询缓存
    比如我有个 sql a 执行时间 2s
    sql b 是基于 sql a 我加了个 and 1==1 执行时间才几毫秒
    两个 sql 的结果是一样的 这是为啥呀
    现在验证不了优化的时间了
    关于     帮助文档     自助推广系统     博客     API     FAQ     Solana     1146 人在线   最高记录 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 31ms UTC 17:49 PVG 01:49 LAX 10:49 JFK 13:49
    Do have faith in what you're doing.
    ubao snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86