GreatSQL社区

搜索

[已解决] tpch 1t 数据导入;lineitem 导入有告警

576 1 2024-3-7 09:34
+ 导入操作如下:

MySQL [tpch]> load data local infile '/home/data_sata/tpchgen/TPC-H_Tools_v3.0.0_1T/dbgen/lineitem.tbl' into table lineitem FIELDS TERMINATED BY '|';
Query OK, 2147482874 rows affected, 65535 warnings (1 day 18 hours 35 min 29.40 sec)
Records: 5999989709  Deleted: 0  Skipped: 3852506835  Warnings: 3410046381

MySQL [tpch]> select count(*) from lineitem;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    13
Current database: tpch

+------------+
| count(*)   |
+------------+
| 2147482874 |
+------------+
1 row in set (13 min 37.16 sec)


+ 其他信息:
在以上操作之后,进一步将数据导入了rapid 引擎;
其后退出过客户端,然后show warnings,结果如下

MySQL [tpch]> show warnings;
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
8 rows in set (0.00 sec)




全部回复(1)
yejr 2024-3-7 11:02:37
需要在执行完后立即查看 warnings ,否则就不是当时的了。

GreatSQL支持并行load data,建议使用该特性提升导入效率,参考: https://greatsql.cn/docs/8032-25 ... -parallel-load.html
cse_engineer

10

主题

0

博客

32

贡献

注册会员

Rank: 2

积分
53

助人为乐(铜)勤学好问(铜)

合作电话:010-64087828

社区邮箱:greatsql@greatdb.com

社区公众号
社区小助手
QQ群
GMT+8, 2024-11-21 19:55 , Processed in 0.019230 second(s), 12 queries , Redis On.
快速回复 返回顶部 返回列表