cse_engineer 发表于 2024-3-7 09:34:04

tpch 1t 数据导入;lineitem 导入有告警

+ 导入操作如下:

MySQL > load data local infile '/home/data_sata/tpchgen/TPC-H_Tools_v3.0.0_1T/dbgen/lineitem.tbl' into table lineitem FIELDS TERMINATED BY '|';
Query OK, 2147482874 rows affected, 65535 warnings (1 day 18 hours 35 min 29.40 sec)
Records: 5999989709Deleted: 0Skipped: 3852506835Warnings: 3410046381

MySQL > select count(*) from lineitem;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    13
Current database: tpch

+------------+
| count(*)   |
+------------+
| 2147482874 |
+------------+
1 row in set (13 min 37.16 sec)


+ 其他信息:
在以上操作之后,进一步将数据导入了rapid 引擎;
其后退出过客户端,然后show warnings,结果如下

MySQL > show warnings;
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
| Warning | 1287 | 'COM_FIELD_LIST' is deprecated and will be removed in a future release. Please use SHOW COLUMNS FROM statement instead |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
8 rows in set (0.00 sec)




yejr 发表于 2024-3-7 11:02:37

需要在执行完后立即查看 warnings ,否则就不是当时的了。

GreatSQL支持并行load data,建议使用该特性提升导入效率,参考: https://greatsql.cn/docs/8032-25/user-manual/5-enhance/5-1-highperf-parallel-load.html
页: [1]
查看完整版本: tpch 1t 数据导入;lineitem 导入有告警