本文简单记录一次实践使用过程,涉及presto-mysql,presto-elasticsearch,文中参数未做注释,请参考官方文档,希望能帮到大家
1 下载安装 presto-0.228
<1>下载
服务端
https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.228/presto-server-0.228.tar.gz
客户端
https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.228/presto-cli-0.228-executable.jar
相关jar
https://repo1.maven.org/maven2/com/facebook/presto/presto-base-jdbc/0.228/presto-base-jdbc-0.228.jar
https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.228/presto-spi-0.228.jar
<2>安装:
1> 解压
tar -zxvf presto-server-0.228.tar.gz
2>创建配置目录 etc etc/catalog
cd presto-server-0.228/
mk dir etc
mkdir etc
mkdir data
cd etc
mkdir catalog
3>创建config.properties 集群配置
在新建的etc目录下
vim config.properties
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=9080
query.max-memory=8GB
query.max-memory-per-node=1GB
query.max-total-memory-per-node=2GB
discovery-server.enabled=true
discovery.uri=http://xinyi:9080
4>创建jvm.config 运行环境配置
在新建的etc目录下
vim jvm.config
-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
5>创建log.properties 日志配置
在新建的etc目录下
vim log.properties
com.facebook.presto=INFO
6>创建node.properties ,节点配置
在新建的etc目录下
vim log.properties
node.environment=production
node.id=ffffffff-ffff-ffff-ffff-fffffffffff1
node.data-dir=/opt/presto-server-0.228/data
7>启动
cd /安装目录/bin
./launcher start 后台启动
./launcher stop停止
./launcher run 前台启动,输出日志
./launcher restart 重启
8>/安装目录/var/log以下位置找到日志文件
launcher.log
server.log
http-request.log
<3> 安装客户端
重命名presto-cli-0.228-executable.jar位presto
mv presto-cli-0.228-executable.jar presto
使用命令:
./presto --server locahost:9080 --catalog mysql--schema test
2 presto-mysql
<1>在 /etc/catalog/目录下创建mysql.properties
connector.name=mysql
connection-url=jdbc:mysql://localhost:3306
connection-user=root
connection-password=root
<2>重启presto-server
/launcher restart
<3>测试
在客户端安装目录执行
./presto --server locahost:9080 --catalog mysql--schema test
presto:es> select * from mysql.test.test;
3 presto-elasticsearch
elasticsearch版本最好与plugin目录下elasticsearch版本一致
<1>在 /安装目录/etc/catalog/目录下创建elasticsearch.properties
connector.name=elasticsearch
#elasticsearch.default-schema=default
elasticsearch.table-description-directory=etc/elasticsearch/
elasticsearch.scroll-size=1000
elasticsearch.scroll-timeout=2s
elasticsearch.request-timeout=2s
elasticsearch.max-request-retries=5
elasticsearch.max-request-retry-time=10s
<2>创建elasticsearch目录
cd /安装目录/etc/
mkdir elasticsearch
cd elasticsearch
<3>定义表的描述文件(自定义表名.json)
cd elasticsearch
vim test.json
{"tableName": "es_test",
"schemaName": "es",
"host": "es-ip地址",
"port": 9300,
"clusterName": "my-application",
"index": "test",
"indexExactMatch": false,
"type": "test",
"columns": [
{
"name": "name",
"type": "varchar",
"jsonPath":"name",
"jsonType":"varchar"
},
{
"name": "age",
"type": "integer",
"jsonPath":"age",
"jsonType":"integer"
}
]
}
<4>重启presto-server
/launcher restart
<5>测试
./presto --server locahost:9080 --catalog elasticsearch --schema es
presto:es> select * from elasticsearch.es.es_test;
name | age
------+-----
HL | 12
HLl | 18
(2 rows)
4 多数据源查询
./presto --server locahost:9080 --catalog elasticsearch --schema es
presto:es> select * from mysql.test.user t left join elasticsearch.es.es_test t1 on t.age=t1.age;