distributed drill on local file system

[Vitalii Diravka]

Hi Mehran,

This is a question for user mailing list.

Looks like there are no issues with it, you can run Drill in distributed
mode on Windows, Linux or MacOS based machines.
It necessary to specify *zk.connect* for Zookeeper hostname and port number
in *drill-override.conf* file and to run *>bin/drillbit.sh start *[1].
But a Hadoop cluster is recommended for this purpose [2], therefore not
sure which issues can arise with this system.

[1]
https://drill.apache.org/docs/starting-drill-in-distributed-mode/#drillbit.sh-command-syntax
[2] https://drill.apache.org/docs/distributed-mode-prerequisites/

Kind regards
Vitalii


On Thu, Aug 16, 2018 at 7:11 PM Mehran Dashti [ BR - PD ] <

Hi,
I wanted to know if it is possible or possible by minimal effort to have
distributed drills that work on local file system of their own?
We do not want to have HDFS as file system?
Thank you in advance.
*Best Regards,*
* [image: LOGO1]*
*Mehran Dashti*
*Product Leader*
*09125902452*

[Abhishek Girish]

I'd also like to add that with the DFS storage plugin configured with local
file system, one will have to make sure all nodes with Drillbits have the
same files (under the same directory structure). If a given query on a
dataset spawns a distributed plan (multiple fragments), then it may fail if
one of the fragments on a remote Drillbit cannot find the file being
referenced by the foreman. Also, this is not something that's been well
tried out and documented, so there may be some surprises.

-Abhishek

Post by Vitalii Diravka
Hi Mehran,
This is a question for user mailing list.
Looks like there are no issues with it, you can run Drill in distributed
mode on Windows, Linux or MacOS based machines.
It necessary to specify *zk.connect* for Zookeeper hostname and port number
in *drill-override.conf* file and to run *>bin/drillbit.sh start *[1].
But a Hadoop cluster is recommended for this purpose [2], therefore not
sure which issues can arise with this system.
[1]
https://drill.apache.org/docs/starting-drill-in-distributed-mode/#drillbit.sh-command-syntax
[2] https://drill.apache.org/docs/distributed-mode-prerequisites/
Kind regards
Vitalii
On Thu, Aug 16, 2018 at 7:11 PM Mehran Dashti [ BR - PD ] <

Hi,
I wanted to know if it is possible or possible by minimal effort to have
distributed drills that work on local file system of their own?
We do not want to have HDFS as file system?
Thank you in advance.
*Best Regards,*
* [image: LOGO1]*
*Mehran Dashti*
*Product Leader*
*09125902452*

[王亮]

I also meet this problem when querying apache2 log in different machines.
How I can test this feature?

Post by Abhishek Girish
I'd also like to add that with the DFS storage plugin configured with local
file system, one will have to make sure all nodes with Drillbits have the
same files (under the same directory structure). If a given query on a
dataset spawns a distributed plan (multiple fragments), then it may fail if
one of the fragments on a remote Drillbit cannot find the file being
referenced by the foreman. Also, this is not something that's been well
tried out and documented, so there may be some surprises.
-Abhishek

Post by Vitalii Diravka
Hi Mehran,
This is a question for user mailing list.
Looks like there are no issues with it, you can run Drill in distributed
mode on Windows, Linux or MacOS based machines.
It necessary to specify *zk.connect* for Zookeeper hostname and port

number

Post by Vitalii Diravka
in *drill-override.conf* file and to run *>bin/drillbit.sh start *[1].
But a Hadoop cluster is recommended for this purpose [2], therefore not
sure which issues can arise with this system.
[1]

https://drill.apache.org/docs/starting-drill-in-distributed-mode/#drillbit.sh-command-syntax

Post by Vitalii Diravka
[2] https://drill.apache.org/docs/distributed-mode-prerequisites/
Kind regards
Vitalii
On Thu, Aug 16, 2018 at 7:11 PM Mehran Dashti [ BR - PD ] <

Hi,
I wanted to know if it is possible or possible by minimal effort to

have

Post by Vitalii Diravka

distributed drills that work on local file system of their own?
We do not want to have HDFS as file system?
Thank you in advance.
*Best Regards,*
* [image: LOGO1]*
*Mehran Dashti*
*Product Leader*
*09125902452*