2017-05-01 19:26:36 UTC
argument for both,
Sorry if this is answered somewhere already. I know in the past, there have
been discussions around using two different readers for Parquet, and
performance gains/losses, issues. etc.
Right now, the store.parquet.use_new_reader is set to false (1.10) and I
was trying to get more information about this and what the eventual roadmap
for drill will be...
So, in the docs I see use_new_reader is set to be not supported in this
release. What I am looking for is a little information on:
- What the two readers are (is one a special drill thing, is the other a
standard reader from the parquet project?)
- What is the eventual goal here... to be able to use and switch between
both? To provide the option? To have code parity with another project?
- Do either of the readers work with Arrow?
- How does Arrow and Parquet readers fit together?
- Will the readers ever converge?
I have other questions too, but these are examples of where I am coming
from. Is there a good starting place for my research on this subject?