[DISCUSS] Deprecation policy in Drill

[Arina Ielchiieva]

Hi all,

when it should be considered OK to remove deprecated options / tables in
Drill?
Some projects mark some notion as deprecated in one release, and then
remove in the next.
Will this policy be ok in Drill?

Here are two latest examples:

1. store.hive.optimize_scan_with_native_readers was introduced for parquet
tables, when native readers support was added for other table type, we had
to come up with better option naming to distinguish between table
types: store.hive.parquet.optimize_scan_with_native_reader and
store.hive.maprdb_json.optimize_scan_with_native_reader.
store.hive.optimize_scan_with_native_readers was deprecated in 1.14 and is
planned to be removed in 1.15.

2. We plan to swap sys.options and sys.options_val tables, then depracte
sys.options_val table in 1.15 and completely remove it in 1.16.

Any thoughts?

Kind regards,
Arina

[weijie tong]

I think we should reserve these deprecated options to let users upgrade
easier. Another solution is if we remove these deprecated ones, we should
add a startup checking to let users know these options are removed .

Post by Arina Ielchiieva
Hi all,
when it should be considered OK to remove deprecated options / tables in
Drill?
Some projects mark some notion as deprecated in one release, and then
remove in the next.
Will this policy be ok in Drill?
1. store.hive.optimize_scan_with_native_readers was introduced for parquet
tables, when native readers support was added for other table type, we had
to come up with better option naming to distinguish between table
types: store.hive.parquet.optimize_scan_with_native_reader and
store.hive.maprdb_json.optimize_scan_with_native_reader.
store.hive.optimize_scan_with_native_readers was deprecated in 1.14 and is
planned to be removed in 1.15.
2. We plan to swap sys.options and sys.options_val tables, then depracte
sys.options_val table in 1.15 and completely remove it in 1.16.
Any thoughts?
Kind regards,
Arina

[Charles Givre]

FWIW, since we seem to have a habit of leaving deprecated items hang around in Drill for some time, I would be in favor of having Drill throw warnings in the next version for use of deprecated items (not just options) and then removing them in version n+2.
— C

Post by weijie tong
I think we should reserve these deprecated options to let users upgrade
easier. Another solution is if we remove these deprecated ones, we should
add a startup checking to let users know these options are removed .

Post by Arina Ielchiieva
Hi all,
when it should be considered OK to remove deprecated options / tables in
Drill?
Some projects mark some notion as deprecated in one release, and then
remove in the next.
Will this policy be ok in Drill?
1. store.hive.optimize_scan_with_native_readers was introduced for parquet
tables, when native readers support was added for other table type, we had
to come up with better option naming to distinguish between table
types: store.hive.parquet.optimize_scan_with_native_reader and
store.hive.maprdb_json.optimize_scan_with_native_reader.
store.hive.optimize_scan_with_native_readers was deprecated in 1.14 and is
planned to be removed in 1.15.
2. We plan to swap sys.options and sys.options_val tables, then depracte
sys.options_val table in 1.15 and completely remove it in 1.16.
Any thoughts?
Kind regards,
Arina

[salim achouche]

Drill is a SQL engine, which means the SQL syntax and associated options
(runtime configuration and session properties) constitute its user facing
APIs (if I may say). When we talk about deprecating and then removing
documented session / configuration properties within the same release, then
what does the versioning scheme mean? are we suggesting that:
- Options are not part of the main Drill APIs?
- It is ok to deprecate and then rename such properties (not remove)?
- It is ok to deprecate and then remove existing properties as long as
there is no change in behavior?

[Arina Ielchiieva]

Hi all,

thanks for your feedback. Looks like the majority here is considering Drill
options and tables as part of API.
Based on that when we deprecate any of such notions we should not remove
them to ease updates and not break backward compatibility.
Though when Drill will move to 2.0 (which is next major version, not ETA
yet) where API might be changed, deprecated notions should be considered to
be removed as well.
Does this sound reasonable?

Kind regards,
Arina

Post by salim achouche
Drill is a SQL engine, which means the SQL syntax and associated options
(runtime configuration and session properties) constitute its user facing
APIs (if I may say). When we talk about deprecating and then removing
documented session / configuration properties within the same release, then
- Options are not part of the main Drill APIs?
- It is ok to deprecate and then rename such properties (not remove)?
- It is ok to deprecate and then remove existing properties as long as
there is no change in behavior?