Discussion:
Writing to the connected data source?
Add Reply
Akshat Jiwan Sharma
2017-08-12 03:43:48 UTC
Reply
Permalink
Raw Message
Hello Everyone,

My use case is that I'd like to be able to use Drill as a proxy for my
data sources, not only for querying the data but also for writing the data.
Is it possible to write data to the connected data store using apache
drill rest api? Or perhaps with a custom function
<https://drill.apache.org/docs/develop-custom-functions-introduction/>?

As I understand there is a CTAS
<http://drill.apache.org/docs/create-table-as-ctas/> function allows us
to create new tables but there are a few restrictions
"You cannot create tables using storage plugins, such as Hive and
HBase, that do not specify a workspace."

Does this mean that there are some storage plugins that do specify a
workspace. I looked at the documentation of mongodb storage plugin and
wasn't sure if it did support workspaces? How can I determine if a
storage plugin is writeable or not?

Thanks,
Akshat
Padma Penumarthy
2017-08-13 04:44:35 UTC
Reply
Permalink
Raw Message
Workspaces are supported only for file system based storage plugins (DFS, S3).
You can create tables in workspaces, by configuring workspaces as writable.
Table can be created as file(s) of type parquet, json, psv, csv and tsv, based
on configuration option “store.format".
For Mongo, creating table is not supported.

Thanks,
Padma
Post by Akshat Jiwan Sharma
Hello Everyone,
My use case is that I'd like to be able to use Drill as a proxy for my data sources, not only for querying the data but also for writing the data.
Is it possible to write data to the connected data store using apache drill rest api? Or perhaps with a custom function <https://drill.apache.org/docs/develop-custom-functions-introduction/>?
As I understand there is a CTAS <http://drill.apache.org/docs/create-table-as-ctas/> function allows us to create new tables but there are a few restrictions
"You cannot create tables using storage plugins, such as Hive and HBase, that do not specify a workspace."
Does this mean that there are some storage plugins that do specify a workspace. I looked at the documentation of mongodb storage plugin and wasn't sure if it did support workspaces? How can I determine if a storage plugin
Loading...