Unable to use Drill JDBC Driver: NoClassDefFoundError ( sys/PStoreProvider ) on startup.

[Martin Senne]

Hi all,

Situation

following the details on
https://drill.apache.org/docs/using-the-jdbc-driver/
I'm trying to connect

- with a simple JDBC Client (based on the code on bottom of
"using-the-jdbc-driver"-page)
- using the Drill JDBC Driver(
*./jars/jdbc-driver/drill-jdbc-all-1.2.0.jar* ) in classpath
- to a locally running embedded drill instance. ( started via
./bin/drill-embedded )
- with connection url:* jdbc:drill:schema=P3.public;zk=local*
- Drill Storage P3 is of type "jdbc" and connects to a PostgreSQL DB.
(Remark: Storage P3 is working properly and can be queried in embedded
Drill shell as expected)

Problem

On startup of the simple JDBC Client, I encounter:

17:36:44 DEBUG FormatPluginConfigBase: Found 0 format plugin configuration
classes: .
17:36:44 DEBUG DrillConfig: DrillConfig object initialized.
[error] (run-main-0) java.lang.NoClassDefFoundError:
oadd/org/apache/drill/exec/store/sys/PStoreProvider
java.lang.NoClassDefFoundError:
*oadd/org/apache/drill/exec/store/sys/PStoreProvider*
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at
oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)

Analysis

- Opening drill-jdbc-all-1.2.0 .jar the path
/oadd/org/apache/drill/exec/... is missing in the jar.
- BTW: The same holds for a freshly built drill-1.3.0-snapshot

Questions

- Is it reasonable (and intented) to connect to an embedded drill
instance via the JDBC client, like I do? If not, what are the alternatives?
- Is it a bug (or maybe unintended), that within ./*exec/jdbc-all/pom.xml
*in the <filter>-section of the shade plugin, "exec/store" (line 408) is
filtered?
<exclude>org/apache/drill/exec/store/**</exclude>

Thx for help in advance,

Martin

[Andries Engelbrecht]

Looks like you are using Drill in embedded mode.
Instead of trying to connect to ZK (not available in embedded mode) rather connect to the drillbit directly.

Example below
sqlline -u jdbc:drill:drillbit=localhost

Basically replace zk=localhost with drillbit=localhost

See if that works for you.

—Andries

Post by Martin Senne
Hi all,
Situation
following the details on
https://drill.apache.org/docs/using-the-jdbc-driver/
I'm trying to connect
- with a simple JDBC Client (based on the code on bottom of
"using-the-jdbc-driver"-page)
- using the Drill JDBC Driver(
*./jars/jdbc-driver/drill-jdbc-all-1.2.0.jar* ) in classpath
- to a locally running embedded drill instance. ( started via
./bin/drill-embedded )
- with connection url:* jdbc:drill:schema=P3.public;zk=local*
- Drill Storage P3 is of type "jdbc" and connects to a PostgreSQL DB.
(Remark: Storage P3 is working properly and can be queried in embedded
Drill shell as expected)
Problem
17:36:44 DEBUG FormatPluginConfigBase: Found 0 format plugin configuration
classes: .
17:36:44 DEBUG DrillConfig: DrillConfig object initialized.
oadd/org/apache/drill/exec/store/sys/PStoreProvider
*oadd/org/apache/drill/exec/store/sys/PStoreProvider*
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at
oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Analysis
- Opening drill-jdbc-all-1.2.0 .jar the path
/oadd/org/apache/drill/exec/... is missing in the jar.
- BTW: The same holds for a freshly built drill-1.3.0-snapshot
Questions
- Is it reasonable (and intented) to connect to an embedded drill
instance via the JDBC client, like I do? If not, what are the alternatives?
- Is it a bug (or maybe unintended), that within ./*exec/jdbc-all/pom.xml
*in the <filter>-section of the shade plugin, "exec/store" (line 408) is
filtered?
<exclude>org/apache/drill/exec/store/**</exclude>
Thx for help in advance,
Martin

[Martin Senne]

Hi Andries, hi all,

great stuff,

drillbit=localhost instead of
zk=local

did the trick.

Could you elaborate a little more, on what the difference is or where to
retrieve additional information / documentation?

Thx

Martin

Post by Andries Engelbrecht
Looks like you are using Drill in embedded mode.
Instead of trying to connect to ZK (not available in embedded mode) rather
connect to the drillbit directly.
Example below
sqlline -u jdbc:drill:drillbit=localhost
Basically replace zk=localhost with drillbit=localhost
See if that works for you.
—Andries

Post by Martin Senne
Hi all,
Situation
following the details on
https://drill.apache.org/docs/using-the-jdbc-driver/
I'm trying to connect
- with a simple JDBC Client (based on the code on bottom of
"using-the-jdbc-driver"-page)
- using the Drill JDBC Driver(
*./jars/jdbc-driver/drill-jdbc-all-1.2.0.jar* ) in classpath
- to a locally running embedded drill instance. ( started via
./bin/drill-embedded )
- with connection url:* jdbc:drill:schema=P3.public;zk=local*
- Drill Storage P3 is of type "jdbc" and connects to a PostgreSQL

DB.

Post by Martin Senne
(Remark: Storage P3 is working properly and can be queried in

embedded

Post by Martin Senne
Drill shell as expected)
Problem
17:36:44 DEBUG FormatPluginConfigBase: Found 0 format plugin

configuration

Post by Martin Senne
classes: .
17:36:44 DEBUG DrillConfig: DrillConfig object initialized.
oadd/org/apache/drill/exec/store/sys/PStoreProvider
*oadd/org/apache/drill/exec/store/sys/PStoreProvider*
at

org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)

Post by Martin Senne
at

org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)

Post by Martin Senne
at

org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)

Post by Martin Senne
at

oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)

Post by Martin Senne
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Analysis
- Opening drill-jdbc-all-1.2.0 .jar the path
/oadd/org/apache/drill/exec/... is missing in the jar.
- BTW: The same holds for a freshly built drill-1.3.0-snapshot
Questions
- Is it reasonable (and intented) to connect to an embedded drill
instance via the JDBC client, like I do? If not, what are the

alternatives?

Post by Martin Senne
- Is it a bug (or maybe unintended), that within

./*exec/jdbc-all/pom.xml

Post by Martin Senne
*in the <filter>-section of the shade plugin, "exec/store" (line 408)

is

Post by Martin Senne
filtered?
<exclude>org/apache/drill/exec/store/**</exclude>
Thx for help in advance,
Martin

[Andries Engelbrecht]

Drill can run in 2 modes - Embedded and Distributed.

In Distributed Mode it uses ZK for keeping track of all the Drillbits in a Drill cluster. When connecting to a a distributed cluster you can either connect direct to a single drill or use ZK to get a list if all drillbits and then get connected to a drillbit in the ZK list to manage the session. Using ZK is preferred as it allows clients to connect to the cluster and any one of the drillbits to distribute the session and foreman load, if one drillbit fails clients can still connect to the cluster.

In Embedded mode ZK is not used to maintain a list of drillbits, and typically ZK is not configured (not needed). In this case you basically have to connect directly to the drillbit. You tried to connect to ZK in embedded mode when no ZK was probably available.

An alternative is to configure a 1 node cluster with a drillbit and ZK, and configure Drill in Distributed mode. This allows you to work with Drill as if it is cluster although a single node. The MapR Drill sandbox is a greta example of this and very handy.

Also note that when using Distributed mode and ZK it is always a good idea to make sure hostnames of ZK and the Drillbits can be resolved by the client, not just the IPs. Also good to validate that a client can reach all the nodes with drillbits and ZK on the network, to avoid a client session being assigned to a drillbit it can’t reach.


—Andries

Post by Martin Senne
Hi Andries, hi all,
great stuff,
drillbit=localhost instead of
zk=local
did the trick.
Could you elaborate a little more, on what the difference is or where to retrieve additional information / documentation?
Thx
Martin
Looks like you are using Drill in embedded mode.
Instead of trying to connect to ZK (not available in embedded mode) rather connect to the drillbit directly.
Example below
sqlline -u jdbc:drill:drillbit=localhost
Basically replace zk=localhost with drillbit=localhost
See if that works for you.
—Andries

Post by Martin Senne
Hi all,
Situation
following the details on
https://drill.apache.org/docs/using-the-jdbc-driver/ <https://drill.apache.org/docs/using-the-jdbc-driver/>
I'm trying to connect
- with a simple JDBC Client (based on the code on bottom of
"using-the-jdbc-driver"-page)
- using the Drill JDBC Driver(
*./jars/jdbc-driver/drill-jdbc-all-1.2.0.jar* ) in classpath
- to a locally running embedded drill instance. ( started via
./bin/drill-embedded )
- with connection url:* jdbc:drill:schema=P3.public;zk=local*
- Drill Storage P3 is of type "jdbc" and connects to a PostgreSQL DB.
(Remark: Storage P3 is working properly and can be queried in embedded
Drill shell as expected)
Problem
17:36:44 DEBUG FormatPluginConfigBase: Found 0 format plugin configuration
classes: .
17:36:44 DEBUG DrillConfig: DrillConfig object initialized.
oadd/org/apache/drill/exec/store/sys/PStoreProvider
*oadd/org/apache/drill/exec/store/sys/PStoreProvider*
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at
oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Analysis
- Opening drill-jdbc-all-1.2.0 .jar the path
/oadd/org/apache/drill/exec/... is missing in the jar.
- BTW: The same holds for a freshly built drill-1.3.0-snapshot
Questions
- Is it reasonable (and intented) to connect to an embedded drill
instance via the JDBC client, like I do? If not, what are the alternatives?
- Is it a bug (or maybe unintended), that within ./*exec/jdbc-all/pom.xml
*in the <filter>-section of the shade plugin, "exec/store" (line 408) is
filtered?
<exclude>org/apache/drill/exec/store/**</exclude>
Thx for help in advance,
Martin