Discussion:
Filter by objectId field in Mongo
Adam Gilmore
2015-01-05 08:15:46 UTC
Permalink
Hi all,

I'm trying to work out how to filter by an objectId field using the Mongo
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.

Is this implemented yet? If not, is there a JIRA item for it already?
Kamesh
2015-01-06 16:08:49 UTC
Permalink
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.

0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where _id
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the Mongo
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it already?
--
Kamesh.
Adam Gilmore
2015-01-06 23:52:32 UTC
Permalink
Hi Kamesh,

Unfortunately it's not on _id - it's on another objectId field we have in
our documents. That seems to work fine with _id but with anything else, it
returns no results.

Any thoughts?


Regards,


*Adam Gilmore*

Director of Technology

***@pharmadata.net.au


+61 421 997 655 (Mobile)

1300 733 876 (AU)

+617 3171 9902 (Intl)


*PharmaData*

Data Intelligence Solutions for Pharmacy

www.PharmaData.net.au <http://www.pharmadata.net.au/>



[image: pharmadata-sig]



*Disclaimer*

This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where _id
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the Mongo
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it already?
--
Kamesh.
Jacques Nadeau
2015-01-07 04:18:44 UTC
Permalink
I believe the current Drill MongoDB plugin is trying to work with MongoDB's
extended JSON [1] at the moment. As such, I think you'd need to write the
filter as myfield.`$oid` = 'id value'

http://docs.mongodb.org/manual/reference/mongodb-extended-json/

Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based json
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we have in
our documents. That seems to work fine with _id but with anything else, it
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where _id
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it already?
--
Kamesh.
Adam Gilmore
2015-01-07 08:44:19 UTC
Permalink
Unfortunately, that didn't work. I tried:

select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;

returns no results - but there are definitely rows with that value in it.
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with MongoDB's
extended JSON [1] at the moment. As such, I think you'd need to write the
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based json
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we have in
our documents. That seems to work fine with _id but with anything else,
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it already?
--
Kamesh.
Kamesh
2015-01-07 16:36:44 UTC
Permalink
Hi Adam
This seems to be a bug. Could you please raise a JIRA for this.
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in it.
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to write
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based
json
Post by Jacques Nadeau
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we have
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
--
Kamesh.
Jacques Nadeau
2015-01-08 16:01:56 UTC
Permalink
I think I know what the problem is. Hanifi recently identified that the
Mongo storage plugin doesn't handle * correctly. Instead of returning a
set of columns for the top level of the mongo record, it returns a single
top-level column called star. DRILL-1692 identifies the problem and it
causes a number of issues when using mongo. Can you try each of the
following to see if they work?

select * from mongo.`connect`.events where `*`.events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;

OR

select events from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;

Thanks,
Jacques
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in it.
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to write
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based
json
Post by Jacques Nadeau
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we have
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips` where
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] , "pop" : 7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using the
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc. but
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
Adam Gilmore
2015-01-09 01:26:02 UTC
Permalink
0: jdbc:drill:> select * from mongo.`connect`.events where
`*`.events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
Query failed: Query failed: Failure validating SQL.
org.eigenbase.util.EigenbaseContextException: From line 1, column 44 to
line 1, column 46: Table '*' not found

Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:> select events from mongo.`connect`.events where
events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+------------+
| events |
+------------+
| null |
+------------+
1 row selected (0.261 seconds)

Strange results there - I played around with the second query and it seems
to be able to return anything but * nicely. So you're probably right.


Regards,


*Adam Gilmore*

Director of Technology

***@pharmadata.net.au


+61 421 997 655 (Mobile)

1300 733 876 (AU)

+617 3171 9902 (Intl)


*PharmaData*

Data Intelligence Solutions for Pharmacy

www.PharmaData.net.au <http://www.pharmadata.net.au/>



[image: pharmadata-sig]



*Disclaimer*

This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Jacques Nadeau
I think I know what the problem is. Hanifi recently identified that the
Mongo storage plugin doesn't handle * correctly. Instead of returning a
set of columns for the top level of the mongo record, it returns a single
top-level column called star. DRILL-1692 identifies the problem and it
causes a number of issues when using mongo. Can you try each of the
following to see if they work?
select * from mongo.`connect`.events where `*`.events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
OR
select events from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
Thanks,
Jacques
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in it.
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to write
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based
json
Post by Jacques Nadeau
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we
have
Post by Adam Gilmore
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from
this
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips`
where
Post by Adam Gilmore
Post by Jacques Nadeau
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] ,
7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using
the
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc.
but
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Post by Adam Gilmore
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
Jacques Nadeau
2015-01-09 02:07:28 UTC
Permalink
I gave the slightly wrong query. I forgot that you need to prefix table
name in case of complex paths (which you had done). Try this:

select * from mongo.`connect`.events e where e.`*`.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;

as well as:

select customerId from mongo.`connect`.events e where e.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
Post by Adam Gilmore
0: jdbc:drill:> select * from mongo.`connect`.events where
`*`.events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
Query failed: Query failed: Failure validating SQL.
org.eigenbase.util.EigenbaseContextException: From line 1, column 44 to
line 1, column 46: Table '*' not found
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:> select events from mongo.`connect`.events where
events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+------------+
| events |
+------------+
| null |
+------------+
1 row selected (0.261 seconds)
Strange results there - I played around with the second query and it seems
to be able to return anything but * nicely. So you're probably right.
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Jacques Nadeau
I think I know what the problem is. Hanifi recently identified that the
Mongo storage plugin doesn't handle * correctly. Instead of returning a
set of columns for the top level of the mongo record, it returns a single
top-level column called star. DRILL-1692 identifies the problem and it
causes a number of issues when using mongo. Can you try each of the
following to see if they work?
select * from mongo.`connect`.events where `*`.events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
OR
select events from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
Thanks,
Jacques
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in
it.
Post by Adam Gilmore
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to write
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based
json
Post by Jacques Nadeau
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we
have
Post by Adam Gilmore
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are
not
Post by Adam Gilmore
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail
and
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
delete the original transmission and its contents. Any unauthorised
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this
communication
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from
this
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage
plugin
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips`
where
Post by Adam Gilmore
Post by Jacques Nadeau
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] ,
7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
On Mon, Jan 5, 2015 at 1:45 PM, Adam Gilmore <
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using
the
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc.
but
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Post by Adam Gilmore
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
Adam Gilmore
2015-01-09 02:10:00 UTC
Permalink
0: jdbc:drill:> select * from mongo.`connect`.events e where
e.`*`.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+--+
| |
+--+
+--+
No rows selected (0.646 seconds)
0: jdbc:drill:> select customerId from mongo.`connect`.events e where
e.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+------------+
| customerId |
+------------+
| {"$oid":"54901607f10c2236769f7b3b"} |
+------------+
1 row selected (0.116 seconds)

Interestingly enough, if I run the second query with "limit 2" (or anything
above 1), it will return the first row then freeze the query.
Post by Jacques Nadeau
I gave the slightly wrong query. I forgot that you need to prefix table
select * from mongo.`connect`.events e where e.`*`.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
select customerId from mongo.`connect`.events e where e.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
Post by Adam Gilmore
0: jdbc:drill:> select * from mongo.`connect`.events where
`*`.events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
Query failed: Query failed: Failure validating SQL.
org.eigenbase.util.EigenbaseContextException: From line 1, column 44 to
line 1, column 46: Table '*' not found
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:> select events from mongo.`connect`.events where
events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+------------+
| events |
+------------+
| null |
+------------+
1 row selected (0.261 seconds)
Strange results there - I played around with the second query and it
seems
Post by Adam Gilmore
to be able to return anything but * nicely. So you're probably right.
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information that
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not the
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised use,
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should check
this email and any attachments for viruses and other defects. The Company
disclaims any liability for loss or damage arising in any way from this
communication including any file attachments.
Post by Jacques Nadeau
I think I know what the problem is. Hanifi recently identified that the
Mongo storage plugin doesn't handle * correctly. Instead of returning a
set of columns for the top level of the mongo record, it returns a
single
Post by Adam Gilmore
Post by Jacques Nadeau
top-level column called star. DRILL-1692 identifies the problem and it
causes a number of issues when using mongo. Can you try each of the
following to see if they work?
select * from mongo.`connect`.events where `*`.events.customerId.`$oid`
=
Post by Adam Gilmore
Post by Jacques Nadeau
'54901607f10c2236769f7b3b' limit 1;
OR
select events from mongo.`connect`.events where
events.customerId.`$oid` =
Post by Adam Gilmore
Post by Jacques Nadeau
'54901607f10c2236769f7b3b' limit 1;
Thanks,
Jacques
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in
it.
Post by Adam Gilmore
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to
write
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb
based
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
json
Post by Jacques Nadeau
object filters.
On Tue, Jan 6, 2015 at 3:52 PM, Adam Gilmore <
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we
have
Post by Adam Gilmore
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain
information
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and
is
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
intended solely for the use of the intended recipient. If you are
not
Post by Adam Gilmore
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail
and
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
delete the original transmission and its contents. Any
unauthorised
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this
communication
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from
this
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage
plugin
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips`
where
Post by Adam Gilmore
Post by Jacques Nadeau
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] ,
7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
On Mon, Jan 5, 2015 at 1:45 PM, Adam Gilmore <
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using
the
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}'
etc.
Post by Adam Gilmore
Post by Jacques Nadeau
but
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Post by Adam Gilmore
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
Adam Gilmore
2015-01-09 01:26:28 UTC
Permalink
0: jdbc:drill:> select * from mongo.`connect`.events where
`*`.events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
Query failed: Query failed: Failure validating SQL.
org.eigenbase.util.EigenbaseContextException: From line 1, column 44 to
line 1, column 46: Table '*' not found

Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:> select events from mongo.`connect`.events where
events.customerId.`$oid` = '54901607f10c2236769f7b3b' limit 1;
+------------+
| events |
+------------+
| null |
+------------+
1 row selected (0.261 seconds)

Strange results there - I played around with the second query and it seems
to be able to return anything but * nicely. So you're probably right.
Post by Jacques Nadeau
I think I know what the problem is. Hanifi recently identified that the
Mongo storage plugin doesn't handle * correctly. Instead of returning a
set of columns for the top level of the mongo record, it returns a single
top-level column called star. DRILL-1692 identifies the problem and it
causes a number of issues when using mongo. Can you try each of the
following to see if they work?
select * from mongo.`connect`.events where `*`.events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
OR
select events from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
Thanks,
Jacques
Post by Adam Gilmore
select * from mongo.`connect`.events where events.customerId.`$oid` =
'54901607f10c2236769f7b3b' limit 1;
returns no results - but there are definitely rows with that value in it.
Post by Jacques Nadeau
I believe the current Drill MongoDB plugin is trying to work with
MongoDB's
Post by Jacques Nadeau
extended JSON [1] at the moment. As such, I think you'd need to write
the
Post by Jacques Nadeau
filter as myfield.`$oid` = 'id value'
http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Note that currently Drill uses SQL expressions with dotted notation
extensions for filters and doesn't currently support the mongodb based
json
Post by Jacques Nadeau
object filters.
Post by Adam Gilmore
Hi Kamesh,
Unfortunately it's not on _id - it's on another objectId field we
have
Post by Adam Gilmore
in
Post by Jacques Nadeau
Post by Adam Gilmore
our documents. That seems to work fine with _id but with anything
else,
Post by Jacques Nadeau
it
Post by Adam Gilmore
returns no results.
Any thoughts?
Regards,
*Adam Gilmore*
Director of Technology
+61 421 997 655 (Mobile)
1300 733 876 (AU)
+617 3171 9902 (Intl)
*PharmaData*
Data Intelligence Solutions for Pharmacy
www.PharmaData.net.au <http://www.pharmadata.net.au/>
[image: pharmadata-sig]
*Disclaimer*
This communication including any attachments may contain information
that
Post by Jacques Nadeau
Post by Adam Gilmore
is either confidential or otherwise protected from disclosure and is
intended solely for the use of the intended recipient. If you are not
the
Post by Jacques Nadeau
Post by Adam Gilmore
intended recipient please immediately notify the sender by e-mail and
delete the original transmission and its contents. Any unauthorised
use,
Post by Jacques Nadeau
Post by Adam Gilmore
dissemination, forwarding, printing, or copying of this communication
including any file attachments is prohibited. The recipient should
check
Post by Jacques Nadeau
Post by Adam Gilmore
this email and any attachments for viruses and other defects. The
Company
Post by Jacques Nadeau
Post by Adam Gilmore
disclaims any liability for loss or damage arising in any way from
this
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
communication including any file attachments.
Post by Kamesh
Hi Adam,
Are you trying apply filter using _id?. If so, Mongo storage plugin
supports it.
0: jdbc:drill:zk=localhost:2181> select * from mongo.usda.`zips`
where
Post by Adam Gilmore
Post by Jacques Nadeau
_id
Post by Adam Gilmore
Post by Kamesh
= '01077';
+------------+
| * |
+------------+
| { "city" : "SOUTHWICK" , "loc" : [ -72.770588 , 42.051099] ,
7667
, "state" : "MA"} |
+------------+
1 row selected (4.792 seconds)
Post by Adam Gilmore
Hi all,
I'm trying to work out how to filter by an objectId field using
the
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Mongo
Post by Adam Gilmore
plugin. I've tried many combinations of = '{$oid: ''id''}' etc.
but
Post by Adam Gilmore
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
Post by Adam Gilmore
nothing seems to work.
Is this implemented yet? If not, is there a JIRA item for it
already?
Post by Jacques Nadeau
Post by Adam Gilmore
Post by Kamesh
--
Kamesh.
Loading...