Discussion:
Drills' Hbase storage plugin
(too old to reply)
Ascot Moss
2017-08-12 08:11:51 UTC
Permalink
Raw Message
Hi,

I have Hbase with Kerberos enabled, how to configure drill's hbase storage
plugin in order to access kerberos-Hbase?

Regards
Ascot Moss
2017-08-12 12:01:09 UTC
Permalink
Raw Message
Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link
about this.

https://drill.apache.org/docs/hbase-storage-plugin/
This link does not provide too much detail about Drills' Hbase Storage on
HBase-Kerberos.
Post by Ascot Moss
Hi,
I have Hbase with Kerberos enabled, how to configure drill's hbase storage
plugin in order to access kerberos-Hbase?
Regards
Sorabh Hamirwasia
2017-08-15 02:14:59 UTC
Permalink
Raw Message
Hi Ascot,

I am not sure if Kerberos between Hbase storage plugin and Hbase server has been tried before or not. Link in [1] talks about Kerberos setup between Drill Client and Drillbit. However looking into the code for Hbase and link at [2], it looks like if you can put a hbase-site.xml in classpath with corresponding setting, then it should work, since internally Drill's hbase storage plugin creates a HBaseConfiguration instance which loads this file in class-path and set corresponding settings for authentication. You need to create TGT for Drill process user (by running kinit, also information provided in link [2]) which will be running as HBase client to the server.


HBaseConnectionKey::getHBaseConf() {

return storeConfig.getHBaseConf();
}


Under HBaseStoragePluginConfig you can find this is how configuration is created:


public Configuration getHBaseConf() {
if (hbaseConf == null) {
hbaseConf = HBaseConfiguration.create();
...

...

}

...

...

}



[1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<https://drill.apache.org/docs/configuring-kerberos-authentication/><https://drill.apache.org/docs/configuring-kerberos-authentication/>
[2]: http://hbase.apache.org/0.94/book/security.html


Thanks,
Sorabh

________________________________
From: Ascot Moss <***@gmail.com>
Sent: Saturday, August 12, 2017 5:01 AM
To: ***@drill.apache.org
Subject: Re: Drills' Hbase storage plugin

Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link
about this.

https://drill.apache.org/docs/hbase-storage-plugin/
This link does not provide too much detail about Drills' Hbase Storage on
HBase-Kerberos.
Post by Ascot Moss
Hi,
I have Hbase with Kerberos enabled, how to configure drill's hbase storage
plugin in order to access kerberos-Hbase?
Regards
Ascot Moss
2017-08-21 07:38:13 UTC
Permalink
Raw Message
Dear Sorabh,

Your reply helps a lot!!

I have copied hbase-site.xml to $DRILL_HOME/conf,
restarted drillbit, ran below:

kinit
sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
***@MY_DOMAIN.COM"

use hbase;
+------+---------------------------------------------------+
| ok | summary |
+------+---------------------------------------------------+
| true | Default schema chnaged to [hbase] |
+------+---------------------------------------------------+


show tables;
(no result, took long time) <=== any idea how to solve it?


Regards
Post by Sorabh Hamirwasia
Hi Ascot,
I am not sure if Kerberos between Hbase storage plugin and Hbase server
has been tried before or not. Link in [1] talks about Kerberos setup
between Drill Client and Drillbit. However looking into the code for Hbase
and link at [2], it looks like if you can put a hbase-site.xml in classpath
with corresponding setting, then it should work, since internally Drill's
hbase storage plugin creates a HBaseConfiguration instance which loads this
file in class-path and set corresponding settings for authentication. You
need to create TGT for Drill process user (by running kinit, also
information provided in link [2]) which will be running as HBase client to
the server.
HBaseConnectionKey::getHBaseConf() {
return storeConfig.getHBaseConf();
}
public Configuration getHBaseConf() {
if (hbaseConf == null) {
hbaseConf = HBaseConfiguration.create();
...
...
}
...
...
}
[1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
https://drill.apache.org/docs/configuring-kerberos-authentication/><http
s://drill.apache.org/docs/configuring-kerberos-authentication/>
[2]: http://hbase.apache.org/0.94/book/security.html
Thanks,
Sorabh
Ascot Moss
2017-08-21 07:50:59 UTC
Permalink
Raw Message
Dear Sorabh,

About putting a hbase-site.xml in classpath, can you please explain a bit
more about this point?

Regards
Post by Ascot Moss
Dear Sorabh,
Your reply helps a lot!!
I have copied hbase-site.xml to $DRILL_HOME/conf,
kinit
sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
use hbase;
+------+---------------------------------------------------+
| ok | summary |
+------+---------------------------------------------------+
| true | Default schema chnaged to [hbase] |
+------+---------------------------------------------------+
show tables;
(no result, took long time) <=== any idea how to solve it?
Regards
Post by Sorabh Hamirwasia
Hi Ascot,
I am not sure if Kerberos between Hbase storage plugin and Hbase server
has been tried before or not. Link in [1] talks about Kerberos setup
between Drill Client and Drillbit. However looking into the code for Hbase
and link at [2], it looks like if you can put a hbase-site.xml in classpath
with corresponding setting, then it should work, since internally Drill's
hbase storage plugin creates a HBaseConfiguration instance which loads this
file in class-path and set corresponding settings for authentication. You
need to create TGT for Drill process user (by running kinit, also
information provided in link [2]) which will be running as HBase client to
the server.
HBaseConnectionKey::getHBaseConf() {
return storeConfig.getHBaseConf();
}
public Configuration getHBaseConf() {
if (hbaseConf == null) {
hbaseConf = HBaseConfiguration.create();
...
...
}
...
...
}
[1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
https://drill.apache.org/docs/configuring-kerberos-authentication/><
https://drill.apache.org/docs/configuring-kerberos-authentication/>
[2]: http://hbase.apache.org/0.94/book/security.html
Thanks,
Sorabh
Sorabh Hamirwasia
2017-08-22 16:56:47 UTC
Permalink
Raw Message
Hi Ascot,

When Drill's HBase storage plugin creates a HBase connection it uses the storage plugin configuration to get all the required configs. The way storage plugin config is formed is based on code I shared with you below or you can look in [1]. If you look into HBaseConfiguration.create() then it first loads all the HBase related configurations provided as (hbase-site.xml / hbase-default.xml) and then override those with Drill's Storage Plugin configs. So the reason of suggesting hbase-site.xml with all proper configuration in classpath was that the security related configurations will be picked up properly from that file and other configuration will be picked from Storage Plugin configuration and hence it should work. Hope it helps.


[1]: https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePluginConfig.java#L91


Thanks,
Sorabh

________________________________
From: Ascot Moss <***@gmail.com>
Sent: Monday, August 21, 2017 12:50:59 AM
To: ***@drill.apache.org
Subject: Re: Drills' Hbase storage plugin

Dear Sorabh,

About putting a hbase-site.xml in classpath, can you please explain a bit
more about this point?

Regards
Post by Ascot Moss
Dear Sorabh,
Your reply helps a lot!!
I have copied hbase-site.xml to $DRILL_HOME/conf,
kinit
sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
use hbase;
+------+---------------------------------------------------+
| ok | summary |
+------+---------------------------------------------------+
| true | Default schema chnaged to [hbase] |
+------+---------------------------------------------------+
show tables;
(no result, took long time) <=== any idea how to solve it?
Regards
Post by Sorabh Hamirwasia
Hi Ascot,
I am not sure if Kerberos between Hbase storage plugin and Hbase server
has been tried before or not. Link in [1] talks about Kerberos setup
between Drill Client and Drillbit. However looking into the code for Hbase
and link at [2], it looks like if you can put a hbase-site.xml in classpath
with corresponding setting, then it should work, since internally Drill's
hbase storage plugin creates a HBaseConfiguration instance which loads this
file in class-path and set corresponding settings for authentication. You
need to create TGT for Drill process user (by running kinit, also
information provided in link [2]) which will be running as HBase client to
the server.
HBaseConnectionKey::getHBaseConf() {
return storeConfig.getHBaseConf();
}
public Configuration getHBaseConf() {
if (hbaseConf == null) {
hbaseConf = HBaseConfiguration.create();
...
...
}
...
...
}
[1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
https://drill.apache.org/docs/configuring-kerberos-authentication/><
https://drill.apache.org/docs/configuring-kerberos-authentication/>
[2]: http://hbase.apache.org/0.94/book/security.html
Thanks,
Sorabh
Loading...