Discussion:
set up Apache Drill on Windows server in distributed mode
(too old to reply)
Divya Gehlot
2017-08-10 08:45:26 UTC
Permalink
Raw Message
Hi,

Has anybody set up Apache Drill cluster in Windows machines/servers?


Thanks,
Divya
Kunal Khatua
2017-08-11 00:20:59 UTC
Permalink
Raw Message
Most people have used Apache Drill on Windows primarily in Embedded mode because no one appears to have tried for more than 1 Drillbit.

That said, you should be able to run Apache Drill in a distributed mode as well, since it is Java-based and would not need to rely on anything more than a Zookeeper.

-----Original Message-----
From: Divya Gehlot [mailto:***@gmail.com]
Sent: Thursday, August 10, 2017 1:45 AM
To: ***@drill.apache.org
Subject: set up Apache Drill on Windows server in distributed mode

Hi,

Has anybody set up Apache Drill c
Divya Gehlot
2017-08-11 02:02:44 UTC
Permalink
Raw Message
Hi,

Even I used in embedded mode and able to set up successfully Keeping that
in mind try to set up in distributed in windows.
I am able to set up Zookeeper and it is running fine .
I started Drill thorugh drillbit.sh start command in git bash
it started successfully.

*Web Console Error :*
but when I try to open web console using http://<one of the ip or host name
of my 3 nodes>:8047
Its giving page cannot be displayed

*SqlLine Errors :*

1.Direct drill command sqlline.bat -u
"jdbc:drill:zk=NameNode:2181,DataNode1:2181,DataNode2:2181"
C:\apache-drill-1.11.0\bin>sqlline.bat -u
"jdbc:drill:drillbit=NameNode:2181,Da
taNode1:2181,DataNode2:2181"
DRILL_ARGS - " -u
jdbc:drill:drillbit=NameNode:2181,DataNode1:2181,DataNode2:218
1"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
org.apache.drill.exec.rpc.RpcException: H ANDSHAKE_COMMUNICATION : Channel
closed /<ip3>:64513 <--> DataNode1/10.20.
6.158:2181. (state=,code=0)
org.apache.drill.exec.rpc
.RpcException: HANDSHAKE_COMMUNICATION : Channel closed /<ip3>:64513 <-->
DataNode1/<ip2>2181.
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnection
Impl.java:167)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(Dril
lJdbc41Factory.java:72)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.ja
va:69)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDri
ver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at
sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
2. Sql line command using Zookeeper by specifying the IPs of the machines :

C:\apache-drill-1.11.0\bin>sqlline -u "jdbc:drill:zk=<ip1>:2181,<ip2>:

:2181<ip3>::2181"

DRILL_ARGS - " -u jdbc:drill:zk=<ip1>:2181,<ip2>:

:2181<ip3>::2181"

Calculating HADOOP_CLASSPATH ...

HBASE_HOME not detected...

Calculating Drill classpath...

ûu (The system cannot find the file specified)

jdbc:drill:zk=<ip1>:2181,<ip2>:2181<ip3>:2181 (The filename,

directory name, or volume label syntax is incorrect)

apache drill 1.11.0

"what ever the mind of man can conceive and believe, drill can query"

sqlline>


3.Strating sqlline by specifying the hostname :

C:\apache-drill-1.11.0\bin>sqlline.bat -u
"jdbc:drill:zk=NameNode,DataNode1,Data

Node2"

DRILL_ARGS - " -u jdbc:drill:zk=NameNode,DataNode1,DataNode2"

HADOOP_HOME not detected...

HBASE_HOME not detected...

Calculating Drill classpath...

ûu (The system cannot find the file specified)

jdbc:drill:zk=NameNode,DataNode1,DataNode2 (The filename, directory name,
or vol

ume label syntax is incorrect)

apache drill 1.11.0

"got drill?"

sqlline>


Note : Removed the actual IP for security purpose.

Appreciate the help .


Thanks,

Divya
Most people have used Apache Drill on Windows primarily in Embedded mode
because no one appears to have tried for more than 1 Drillbit.
That said, you should be able to run Apache Drill in a distributed mode as
well, since it is Java-based and would not need to rely on anything more
than a Zookeeper.
-----Original Message-----
Sent: Thursday, August 10, 2017 1:45 AM
Subject: set up Apache Drill on Windows server in distributed mode
Hi,
Has anybody set up Apache Drill cluster in Windows machines/servers?
Thanks,
Divya
Kunal Khatua
2017-08-11 05:50:48 UTC
Permalink
Raw Message
If you were able to start Drill correctly in distributed mode, then it should have registered with Zookeeper.
To confirm his, you can look at the Drillbit logs.

You should see something like this, indicating the port number (0.0.0.0:8047):

2017-08-09 22:54:22,253 kk127 [main] INFO org.eclipse.jetty.server.Server - jetty-9.1.5.v20140505
2017-08-09 22:54:23,300 kk127 [main] INFO o.e.j.server.handler.ContextHandler - Started ***@7ebcdf3{/,null,AVAILABLE}
2017-08-09 22:54:23,307 kk127 [main] INFO o.e.jetty.server.ServerConnector - Started ***@13c8e042{HTTP/1.1}{0.0.0.0:8047}
2017-08-09 22:54:23,307 kk127 [main] INFO org.eclipse.jetty.server.Server - Started @11616ms
2017-08-09 22:54:23,308 kk127 [main] INFO o.apache.drill.exec.server.Drillbit - Startup completed (4863 ms).

As for the connection via the SQLLine command... do you have the ZKs running on your "NameNode", "DataNode1", "DataNode2" servers?

The way the distributed mode works is that during startup, the Drillbits register with the Zookeeper, while a JDBC client like SQLLine would look up the list of Drillbits in ZK to connect to one of them.... or directly connect to the Drillbit (default port in this case is 31010).

You'll need to figure out which of these are out of line in your settings.

-----Original Message-----
From: Divya Gehlot [mailto:***@gmail.com]
Sent: Thursday, August 10, 2017 7:03 PM
To: ***@drill.apache.org
Subject: Re: set up Apache Drill on Windows server in distributed mode

Hi,

Even I used in embedded mode and able to set up successfully Keeping that in mind try to set up in distributed in windows.
I am able to set up Zookeeper and it is running fine .
I started Drill thorugh drillbit.sh start command in git bash it started successfully.

*Web Console Error :*
but when I try to open web console using http://<one of the ip or host name of my 3 nodes>:8047 Its giving page cannot be displayed

*SqlLine Errors :*

1.Direct drill command sqlline.bat -u
"jdbc:drill:zk=NameNode:2181,DataNode1:2181,DataNode2:2181"
C:\apache-drill-1.11.0\bin>sqlline.bat -u
"jdbc:drill:drillbit=NameNode:2181,Da
taNode1:2181,DataNode2:2181"
DRILL_ARGS - " -u
jdbc:drill:drillbit=NameNode:2181,DataNode1:2181,DataNode2:218
1"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
Channel closed /<ip3>:64513 <--> DataNode1/10.20.
6.158:2181. (state=,code=0)
org.apache.drill.exec.rpc
.RpcException: HANDSHAKE_COMMUNICATION : Channel closed /<ip3>:64513
<--> DataNode1/<ip2>2181.
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnection
Impl.java:167)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(Dril
lJdbc41Factory.java:72)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.ja
va:69)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDri
ver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at
sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
2. Sql line command using Zookeeper by specifying the IPs of the machines :

C:\apache-drill-1.11.0\bin>sqlline -u "jdbc:drill:zk=<ip1>:2181,<ip2>:

:2181<ip3>::2181"

DRILL_ARGS - " -u jdbc:drill:zk=<ip1>:2181,<ip2>:

:2181<ip3>::2181"

Calculating HADOOP_CLASSPATH ...

HBASE_HOME not detected...

Calculating Drill classpath...

ûu (The system cannot find the file specified)

jdbc:drill:zk=<ip1>:2181,<ip2>:2181<ip3>:2181 (The filename,

directory name, or volume label syntax is incorrect)

apache drill 1.11.0

"what ever the mind of man can conceive and believe, drill can query"

sqlline>


3.Strating sqlline by specifying the hostname :

C:\apache-drill-1.11.0\bin>sqlline.bat -u "jdbc:drill:zk=NameNode,DataNode1,Data

Node2"

DRILL_ARGS - " -u jdbc:drill:zk=NameNode,DataNode1,DataNode2"

HADOOP_HOME not detected...

HBASE_HOME not detected...

Calculating Drill classpath...

ûu (The system cannot find the file specified)

jdbc:drill:zk=NameNode,DataNode1,DataNode2 (The filename, directory name, or vol

ume label syntax is incorrect)

apache drill 1.11.0

"got drill?"

sqlline>


Note : Removed the actual IP for security purpose.

Appreciate the help .


Thanks,

Divya
Most people have used Apache Drill on Windows primarily in Embedded
mode because no one appears to have tried for more than 1 Drillbit.
That said, you should be able to run Apache Drill in a distributed
mode as well, since it is Java-based and would not need to rely on
anything more than a Zookeeper.
-----Original Message-----
Sent: Thursday, August 10, 2017 1:45 AM
Subject: set up Apache Drill on Windows server in distributed mode
Hi,
Has anybody set up Apache Drill cluster in Windows machines/servers?
Thanks,
Divya Gehlot
2017-08-14 02:11:14 UTC
Permalink
Raw Message
Hi ,
When I start the Drill bit using drillbit.sh in got bash as there is no
.cmd file for Drill to start in distributed mode .
When I start there is no error in console.
As advised when i checked the drillbit.out file in log directory .
Only following lines were there
Error: Could not find or load main class
org.apache.drill.exec.server.Drillbit
To confirm that Zookeeper started correctly in distributed mode , I ran
zkCli.cmd command
Below is the stacktrace from one of the node :

Welcome to ZooKeeper!
2017-08-13 21:05:58,589 [myid:] - INFO
socket con
nection to server<hostname>/0:0:0:0:0:0:0
:1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-08-13 21:05:58,596 [myid:] - INFO
connection e
stablished to<hostname>/0:0:0:0:0:0:0:1:2
181, initiating session
JLine support is enabled
2017-08-13 21:05:58,624 [myid:] - INFO
establishm
0:0:0:1:2181, sessionid = 0x25dde7cd51e0000, negotiated timeout = 30000
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]
*Note *- removed the original host name
What else can I check to confirm Drill is configured correctly in
distributed mode ?

Appreciate the help!

Thanks,
Divya
If you were able to start Drill correctly in distributed mode, then it
should have registered with Zookeeper.
To confirm his, you can look at the Drillbit logs.
You should see something like this, indicating the port number (
2017-08-09 22:54:22,253 kk127 [main] INFO org.eclipse.jetty.server.Server
- jetty-9.1.5.v20140505
2017-08-09 22:54:23,300 kk127 [main] INFO o.e.j.server.handler.ContextHandler
2017-08-09 22:54:23,307 kk127 [main] INFO o.e.jetty.server.ServerConnector
2017-08-09 22:54:23,307 kk127 [main] INFO org.eclipse.jetty.server.Server
2017-08-09 22:54:23,308 kk127 [main] INFO o.apache.drill.exec.server.Drillbit
- Startup completed (4863 ms).
As for the connection via the SQLLine command... do you have the ZKs
running on your "NameNode", "DataNode1", "DataNode2" servers?
The way the distributed mode works is that during startup, the Drillbits
register with the Zookeeper, while a JDBC client like SQLLine would look up
the list of Drillbits in ZK to connect to one of them.... or directly
connect to the Drillbit (default port in this case is 31010).
You'll need to figure out which of these are out of line in your settings.
-----Original Message-----
Sent: Thursday, August 10, 2017 7:03 PM
Subject: Re: set up Apache Drill on Windows server in distributed mode
Hi,
Even I used in embedded mode and able to set up successfully Keeping that
in mind try to set up in distributed in windows.
I am able to set up Zookeeper and it is running fine .
I started Drill thorugh drillbit.sh start command in git bash it started successfully.
*Web Console Error :*
but when I try to open web console using http://<one of the ip or host
name of my 3 nodes>:8047 Its giving page cannot be displayed
*SqlLine Errors :*
1.Direct drill command sqlline.bat -u
"jdbc:drill:zk=NameNode:2181,DataNode1:2181,DataNode2:2181"
C:\apache-drill-1.11.0\bin>sqlline.bat -u
"jdbc:drill:drillbit=NameNode:2181,Da
taNode1:2181,DataNode2:2181"
DRILL_ARGS - " -u
jdbc:drill:drillbit=NameNode:2181,DataNode1:2181,DataNode2:218
1"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
Channel closed /<ip3>:64513 <--> DataNode1/10.20.
6.158:2181. (state=,code=0)
org.apache.drill.exec.rpc
.RpcException: HANDSHAKE_COMMUNICATION : Channel closed /<ip3>:64513
<--> DataNode1/<ip2>2181.
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnection
Impl.java:167)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(Dril
lJdbc41Factory.java:72)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.ja
va:69)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDri
ver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.
java:167)
at
sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
:2181<ip3>::2181"
:2181<ip3>::2181"
Calculating HADOOP_CLASSPATH ...
HBASE_HOME not detected...
Calculating Drill classpath...
ûu (The system cannot find the file specified)
jdbc:drill:zk=<ip1>:2181,<ip2>:2181<ip3>:2181 (The filename,
directory name, or volume label syntax is incorrect)
apache drill 1.11.0
"what ever the mind of man can conceive and believe, drill can query"
sqlline>
C:\apache-drill-1.11.0\bin>sqlline.bat -u "jdbc:drill:zk=NameNode,
DataNode1,Data
Node2"
DRILL_ARGS - " -u jdbc:drill:zk=NameNode,DataNode1,DataNode2"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
ûu (The system cannot find the file specified)
jdbc:drill:zk=NameNode,DataNode1,DataNode2 (The filename, directory name, or vol
ume label syntax is incorrect)
apache drill 1.11.0
"got drill?"
sqlline>
Note : Removed the actual IP for security purpose.
Appreciate the help .
Thanks,
Divya
Most people have used Apache Drill on Windows primarily in Embedded
mode because no one appears to have tried for more than 1 Drillbit.
That said, you should be able to run Apache Drill in a distributed
mode as well, since it is Java-based and would not need to rely on
anything more than a Zookeeper.
-----Original Message-----
Sent: Thursday, August 10, 2017 1:45 AM
Subject: set up Apache Drill on Windows server in distributed mode
Hi,
Has anybody set up Apache Drill cluster in Windows machines/servers?
Thanks,
Divya
Loading...