wangyu8460958 最近的时间轴更新 wangyu8460958 最近的时间轴更新 | wangyu8460958V2EX 第 227278 号会员，加入于 2017-04-22 10:01:56 +08:00 |
| Python 队列脚本执行时报错 Warning: Truncated incorrect DOUBLE value: '[2627290]' r = r + self.execute(query, a) Python wangyu8460958 2017 年 5 月 5 日 |
| 定义 1 个数组 a，它的长度为 1000,不设置初始值，应该如何来写？ 编程 wangyu8460958 2017 年 4 月 27 日 |
| 请问下在 Python 中如何将一个数组的值赋给另外一个数组 问与答 wangyu8460958 2017 年 4 月 25 日 最后回复来自 konia | 1 |
| Python 如何分批次导入 100W 条 mysql 数据 Python wangyu8460958 2017 年 4 月 27 日 最后回复来自 wangyu8460958 | 4 |
wangyu8460958 最近回复了
2017 年 4 月 27 日 回复了 wangyu8460958 创建的主题 Python Python 如何分批次导入 100W 条 mysql 数据 |
下面是我自己写的脚本，下面的脚本还能够优化吗？
#coding=utf-8
import MySQLdb
import numpy as np
cOnn= MySQLdb.connect(host='localhost',port = 3306,user='root',passwd='123456',db='google')
cur = conn.cursor()
conn.autocommit(1)
query_sql = "select id from google.Video where created_at < date_sub(now(), interval 1 year);"
insert_sql = "insert ignore into tmp.Video_2017bak (select * from google.Video where id = %s);"
delete_sql = "delete from google.Video where id = %s;"
cur.execute(query_sql)
dataList = cur.fetchall()
aaa = np.array(dataList)
ids = []
for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(insert_sql,ids)
ids = []
cur.executemany(insert_sql,ids)
ids = []
for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(delete_sql,ids)
ids = []
cur.executemany(delete_sql,ids)
ids = []
cur.close()
conn.close()
#coding=utf-8
import MySQLdb
import numpy as np
cOnn= MySQLdb.connect(host='localhost',port = 3306,user='root',passwd='123456',db='google')
cur = conn.cursor()
conn.autocommit(1)
query_sql = "select id from google.Video where created_at < date_sub(now(), interval 1 year);"
insert_sql = "insert ignore into tmp.Video_2017bak (select * from google.Video where id = %s);"
delete_sql = "delete from google.Video where id = %s;"
cur.execute(query_sql)
dataList = cur.fetchall()
aaa = np.array(dataList)
ids = []
for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(insert_sql,ids)
ids = []
cur.executemany(insert_sql,ids)
ids = []
for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(delete_sql,ids)
ids = []
cur.executemany(delete_sql,ids)
ids = []
cur.close()
conn.close()
2017 年 4 月 22 日 回复了 wangyu8460958 创建的主题 Python Python 如何分批次导入 100W 条 mysql 数据 |
@skydiver 我想使用脚本让它每周执行一次，所以想使用 python 脚本来做。
Select Language