hive导入动态分区出现错误couldonlybereplicatedto0nodes,insteadof1怎么解决
这篇文章主要介绍“hive导入动态分区出现错误could only be replicated to 0 nodes, instead of 1怎么解决”,在日常操作中,相信很多人在hive导入动态分区出现错误could only be replicated to 0 nodes, instead of 1怎么解决问题上存在疑惑,小编查阅了各式资料,整理出简单好用的操作方法,希望对大家解答”hive导入动态分区出现错误could only be replicated to 0 nodes, instead of 1怎么解决”的疑惑有所帮助!接下来,请跟着小编一起来学习吧!
创新互联建站专注于企业营销型网站建设、网站重做改版、凭祥网站定制设计、自适应品牌网站建设、成都h5网站建设、购物商城网站建设、集团公司官网建设、外贸营销网站建设、高端网站制作、响应式网页设计等建站业务,价格优惠性价比高,为凭祥等各大城市提供网站开发制作服务。
insert overwrite table default.`table` partition(part) select t.*,t.detailid AS part from default.`table` t limit 10 ;
这个表有4000多万数据
异常:
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/hadoop/dynamic/.hive-staging_hive_2021-04-28_10-54-00_523_4426014102702927767-1/_task_tmp.-ext-10000/part=96307/_tmp.000012_3 could only be replicated to 0 nodes instead of minReplication (=1). There are 5 datanode(s) running and no node(s) are excluded in this operation. at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1731) at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:265) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2567) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:829) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:510) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:850) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:793) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2489)
发现磁盘好像快满了。
改成导入1000万条成功
http://wenda.chinahadoop.cn/question/3964
https://blog.csdn.net/zhongxiayimeng/article/details/73188511?utm_medium=distribute.pc_relevant_download.none-task-blog-baidujs-1.nonecase&depth_1-utm_source=distribute.pc_relevant_download.none-task-blog-baidujs-1.nonecase
到此,关于“hive导入动态分区出现错误could only be replicated to 0 nodes, instead of 1怎么解决”的学习就结束了,希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习,快去试试吧!若想继续学习更多相关知识,请继续关注创新互联网站,小编会继续努力为大家带来更多实用的文章!
当前文章:hive导入动态分区出现错误couldonlybereplicatedto0nodes,insteadof1怎么解决
URL地址:http://scyanting.com/article/gopoij.html