quickSQL往spark上提交的问题

[tb77506668]

Versions = 0.7.0

启动:
./bin/quicksql-server.sh start -p 5888 -r spark -m yarn-client

显示命令:
nohup /usr/hdp/3.1.0.0-78/spark2/bin/spark-submit --jars /app/qsql-0.7.0/lib/sqlite-jdbc-.jar,/app/qsql-0.7.0/lib/qsql-meta-0.7.0.jar,/app/qsql-0.7.0/lib/jackson-dataformat-cbor-.jar,/app/qsql-0.7.0/lib/jackson-dataformat-smile-.jar,/app/qsql-0.7.0/lib/qsql-client-0.7.0.jar,/app/qsql-0.7.0/lib/jetty-http-.jar,/app/qsql-0.7.0/lib/jetty-io-.jar,/app/qsql-0.7.0/lib/jetty-security-.jar,/app/qsql-0.7.0/lib/jetty-server-.jar,/app/qsql-0.7.0/lib/jetty-util-.jar,/app/qsql-0.7.0/lib/commons-cli-.jar,/app/qsql-0.7.0/lib/avatica-core-.jar,/app/qsql-0.7.0/lib/avatica-server-.jar,/app/qsql-0.7.0/lib/avatica-metrics-.jar,/app/qsql-0.7.0/lib/protobuf-java-.jar,/app/qsql-0.7.0/lib/jackson-core-.jar,/app/qsql-0.7.0/lib/jackson-annotations-.jar,/app/qsql-0.7.0/lib/jackson-databind-.jar,/app/qsql-0.7.0/lib/httpclient-.jar,/app/qsql-0.7.0/lib/httpcore-.jar,/app/qsql-0.7.0/lib/esri-geometry-api-.jar,/app/qsql-0.7.0/lib/guava-.jar,/app/qsql-0.7.0/lib/derby-.jar,/app/qsql-0.7.0/lib/jackson-dataformat-yaml-.jar,/app/qsql-0.7.0/lib/imc-.jar,/app/qsql-0.7.0/lib/qsql-core-0.7.0.jar,/app/qsql-0.7.0/lib/qsql-calcite-parser-0.7.0.jar,/app/qsql-0.7.0/lib/elasticsearch-rest-client-.jar,/app/qsql-0.7.0/lib/httpasyncclient-.jar,/app/qsql-0.7.0/lib/httpclient-.jar,/app/qsql-0.7.0/lib/httpcore-.jar,/app/qsql-0.7.0/lib/httpcore-nio-.jar,/app/qsql-0.7.0/lib/elasticsearch-spark-.jar,/app/qsql-0.7.0/lib/json-path-.jar,/app/qsql-0.7.0/lib/snakeyaml-.jar,/app/qsql-0.7.0/lib/mysql-connector-java-.jar,/app/qsql-0.7.0/lib/ojdbc6-.jar,/app/qsql-0.7.0/lib/postgresql-.jar --master yarn-client --executor-memory 1G --driver-memory 2G --num-executors 10 --conf spark.driver.userClassPathFirst=true --class com.qihoo.qsql.server.JdbcServer /app/qsql-0.7.0/lib/qsql-server-0.7.0.jar --port 5888 > /app/qsql-0.7.0/logs/stdout.log 2>&1 &

public static void main(String[] args) {
    try {
        Class.forName("com.qihoo.qsql.client.Driver");
        Properties properties = new Properties();
        properties.setProperty("runner","spark");
        properties.setProperty("appName","quicksql");
        String url = "jdbc:quicksql:url=http://192.168.62.37:5888";
        Connection connection = DriverManager.getConnection(url, properties);
        Statement statement = connection.createStatement();
        String sql = "select city_name ,count(city_name) pv, sum(pv)  num from qsql_city q join ods.quicksql_test  t on q.city_id = t.city_id group by city_name";

        ResultSet resultSet = statement.executeQuery(sql);
        while (resultSet.next()){
            System.out.println(String.format("%s %s %s",resultSet.getString(1),resultSet.getString(2),resultSet.getString(3)));
        }

    } catch (ClassNotFoundException e) {
        e.printStackTrace();
    } catch (SQLException e) {
        e.printStackTrace();
    }
}

使用spark模式,但是并没有提交到yarn上(用的yarn-client模式)
使用 ./bin/quicksql.sh -e "select count(1) from ods.quicksql_test" 是可以使用spark提交到yarn上的

[FishermanZzhang]

+1