Discussion:
System Error PhysicalOperatorSetupException: Can not parallelize fragment
(too old to reply)
Padma Penumarthy
2017-07-31 21:15:02 UTC
Permalink
Raw Message
Looking at the log you provided, seems like you are working on indexr plugin ? You probably have a groupScan implemented for the plugin.


What this error is saying is

you want to have minor fragments on 2 nodes you have (slave1 and slave2) because they are marked as mandatory , however you configured your overall max parallelization as only 1.


You should check how you are configuring/calculating endpoint affinity for your groupScan. Both endpoints are marked as mandatory. Also, for slave2, the values do not look correct. Typically, affinity value should be <= 1.


EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port: 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647]


Overall max parallelization is obtained from getMaxParallelizationWidth() you implemented for your groupScan. Seems like you are returning 1 for that which contradicts with how you are configuring the endpoint affinities.


You can look at other groupScan implementations (Parquet or HBase) to see how this should be done.



Thanks,

Padma

________________________________
From: ºÎœšŸü <***@163.com>
Sent: Sunday, July 30, 2017 6:33:04 PM
To: ***@drill.apache.org
Subject: System Error PhysicalOperatorSetupException: Can not parallelize fragment

Hi , nice to meet you .
thinks to read this email.
when used drill to Query, meets some error very confused me, and i don't know how to solve it ,hope you could check and help me thinks a lot .


to find any useful message ,so i modify some class to print error.
here is my modify :
drill-java-exec-1.9.0.jar
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.java
line 80
from
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is more than the allowed fragment max " +
"width ({}).", endpointPool.size(), pInfo.getMaxWidth());
to
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is more than the allowed fragment max " +
"width ({}), fragment: {}, endpointPool: {}, pInfo.getEndpointAffinityMap(): {}.",
endpointPool.size(),
pInfo.getMaxWidth(),
fragmentWrapper.getNode(),
org.apache.commons.lang3.StringUtils.join( endpointPool.values(), ","),
org.apache.commons.lang3.StringUtils.join(pInfo.getEndpointAffinityMap().values(), ","));


here is the meesage of error log




[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.




[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) [drill-java-exec-1.9.0.jar:1.9.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can not parallelize fragment.
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
2017-07-27 12:01:15,912 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 26869a74-53da-2307-1e8d-9326be0d04eb: select sum(t.num) num from (select province,count(distinct phone) as num from indexr.count_phone where ctime <= 20170725 group by province) t
2017-07-27 12:01:16,100 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 6 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 33hjj=======================maxPw=: 1
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 44hjj=======================maxPw=: 1001
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - 000000============width=2
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - 111111============width=1 pInfo.getMaxWidth()=1
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory endpoints (2) that require an assignment is more than the allowed fragment max width (1), fragment: FragmentNode [root=Screen [endpoint=address: "slave1"
user_port: 31010
control_port: 31011
data_port: 31012
, getChild()=***@ae9bd17], sendingExchange=null, receivingExchangePairs=[]], endpointPool: EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port: 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647], pInfo.getEndpointAffinityMap(): EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port: 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647].
2017-07-27 12:01:16,160 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.




[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.




[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) [drill-java-exec-1.9.0.jar:1.9.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can not parallelize fragment.
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted

Loading...