场景的简要概述:
我们在船上有一个数据记录系统,各种传感器正在实时读取数据,并将数据存储在MySQL数据库中。
每个传感器都有一个表,其中的瞬时传感器值有时间戳并存储在数据库中。
现在的要求是将所有传感器的数据合并到一个表中,表中的值在两个日期时间值之间,每分钟平均一次。
以下是我迄今为止所做的:
一。创建存储过程以创建日历表。
calendar过程创建一个表,其中日期时间戳位于两个指定的日期时间值之间。
对于巡航报告,我正在处理日历表,如下所示:
cal
-------------------+
dt
-------------------+
2012-07-09 00:00:00
2012-07-09 00:01:00
2012-07-09 00:02:00
... etc
2012-07-29 23:57:00
2012-07-29 23:58:00
2012-07-29 23:59:00
总共有30241条记录,在0.016秒内提取,所以没有问题。
2。为每分钟的平均传感器值创建临时表。
平均传感器表示例:
tbl_gyro_hdt_1min_ave
-------------------+------------------
tmstamp | average_heading
-------------------+------------------
2012-07-09 00:00:00, 135.633333333333
2012-07-09 00:01:00, 135.633333333333
2012-07-09 00:02:00, 136.1
2012-07-09 00:03:00, 135.433333333333
etc...
29546 records fetched in 0.047 secs
还有另一个传感器表:
tbl_par_sensor_1min_ave
-------------------+------------------
tmstamp | average_par
-------------------+------------------
2012-07-09 00:00:00, 16.269949
2012-07-09 00:01:00, 16.270832
2012-07-09 00:02:00, 16.2637752
2012-07-09 00:03:00, 16.2678025
2012-07-09 00:04:00, 16.269324
2012-07-09 00:05:00, 16.2721382
etc...
29543 records fetched in 0.047 secs
三。现在,将临时表连接到日历表是轮子脱落的地方。
要将单个表联接到日历表,请执行以下操作:
SELECT cal.dt, tbl_gyro_hdt_1min_ave.average_heading
FROM cal
LEFT JOIN tbl_gyro_hdt_1min_ave
ON cal.dt = tbl_gyro_hdt_1min_ave.tmstamp
以上问题说明:
+----+---------------+-----------------------+--------+---------------+-------+---------+------+-------+-------------+
| Id | Select_Type | Table | Type | Possible_Keys | Key | Key_Len | Ref | Rows | Extra |
+----+---------------+-----------------------+--------+---------------+-------+---------+------+-------+-------------+
| 1 | SIMPLE | cal | index | NULL | dt | 9 | NULL | 30243 | Using index |
| 1 | SIMPLE | tbl_gyro_hdt_1min_ave | ALL | date_index | NULL | NULL | NULL | 29546 | |
+----+---------------+-----------------------+--------+---------------+-------+---------+------+-------+-------------+
对于非常小的数据集,这很好,但对于上面的示例,它只是挂起。
我试着给所有的表添加索引,结果都一样。
编辑>我让整个数据集在一夜之间运行。
结果是:
获取了30243条记录。
持续时间:23.697秒,3000.352秒
下一步是将两个以上的表与日历表相关联,如下所示:
SELECT cal.dt, tbl_par_sensor_1min_ave.average_par, tbl_gyro_hdt_1min_ave.average_heading
FROM tbl_par_sensor_1min_ave
LEFT JOIN cal
ON cal.dt = tbl_par_sensor_1min_ave.tmstamp
LEFT JOIN tbl_gyro_hdt_1min_ave
ON cal.dt = tbl_gyro_hdt_1min_ave.tmstamp
毫不奇怪,这个也挂着。
任何指点都非常感谢。
根据以下注释的要求,以下是表架构:
show columns from cal;
+-------+----------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+----------+------+-----+---------+-------+
| dt | datetime | YES | MUL | NULL | |
+-------+----------+------+-----+---------+-------+
1 row in set (0.00 sec)
show columns from tbl_gyro_hdt_1min_ave;
+-----------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+-------------+------+-----+---------+-------+
| tmstamp | varchar(24) | YES | MUL | NULL | |
| average_heading | double | YES | | NULL | |
+-----------------+-------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
show columns from tbl_par_sensor_1min_ave;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| tmstamp | varchar(24) | YES | MUL | NULL | |
| average_par | double | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
解决了的:
在实现setsuna的更改之后:
单个外部连接:
SELECT cal.dt, tbl_gyro_hdt_1min_ave.average_heading
FROM cal
LEFT JOIN tbl_gyro_hdt_1min_ave
ON cal.dt = tbl_gyro_hdt_1min_ave.tmstamp
Fetched 30243 records
Duration: 0.015 sec
Fetched in: 0.172 sec
双外部连接:
SELECT cal.dt, tbl_gyro_hdt_1min_ave.average_heading, tbl_par_sensor_1min_ave.average_par
FROM cal
LEFT JOIN tbl_gyro_hdt_1min_ave
ON cal.dt = tbl_gyro_hdt_1min_ave.tmstamp
LEFT JOIN tbl_par_sensor_1min_ave
ON cal.dt = tbl_par_sensor_1min_ave.tmstamp
Fetched 29543 records
Duration: 0.000s
Fetched in: 0.281 sec
最佳答案
将列cal.dt更改为不空,并将tmstamp同时更改为TIMESTAMP或DATETIME且不为空。一个包含约30000条记录和正确索引的连接条件字段的连接应该运行得相当快。
注意:@Knapie已经给出了这个答案的结果。