博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
hive常见问题解决干货大全
阅读量:4983 次
发布时间:2019-06-12

本文共 8441 字,大约阅读时间需要 28 分钟。

 

  本人,苦心多时,历经磨难和心血,与大家共同攻克问题难关!

 

问题一: 

  FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

 参考:http://blog.csdn.net/l1028386804/article/details/51566262

 解决办法:

  进入所安装的Hive的conf目录,找到hive-site.xml,(若没修改,则是hive-default.xml.template)。

<property>

  <name>hive.metastore.schema.verification</name>

  <value>true</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

 

改为

<property>

  <name>hive.metastore.schema.verification</name>

  <value>false</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

  然后,再重启hive,进入即可。

   或者  在hive安装目录下的bin目录下,./hive -hiveconf hive.root.logger=DEBUG,console

 

  同时,最近还有人来问我。出现如下问题SeesionHiveMetaStoreClient

  先按照我的问题一的思路去解决,若解决不了,看看你的hdfs-site.xml和core-site.xml是否正确。(因为有人问我,最后帮解决发现是这个问题)

见。如下博客

   和

http://www.cnblogs.com/braveym/p/6685045.html

 

 

 

 

 

 

 

  问题二:

    Hive报错为:There are 2 datanode(s) running and no node(s) are excluded in this operation.

  详细报错为:

 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/zhou/ed69eb4e-855a-4b18-a498-345e8685533a/hive_2015-12-14_05-32-03_335_9179215496953152524-1/-mr-10004/f8d2e938-b439-495f-b886-801c98f660da/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and no node(s) are excluded in this operation.

  参考:http://jingyan.baidu.com/article/7082dc1c65a76be40a89bd09.html

  解决方法:

      出现此类报错主要原因是datanode存在问题,要么硬盘容量不够,要么datanode服务器down了。检查datanode,重启Hadoop即可解决。

 

 

  问题三:

   hive保持,  Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask

  参考:http://blog.sina.com.cn/s/blog_75353ff40102v0d3.html

  解决:

      ./hive -hiveconf hive.root.logger=DEBUG,console
    准备通过控制台输出来定位出错位置,没想到这样设置后,运行程序就成功了。自己比较莫名其妙。
    总之还是要养成常看错误日志解决问题。

 

   问题四:

 

Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema.  If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))  

这是由于没有初始化Hive元数据的数据库,默认情况下,Hive的元数据保存在了内嵌的derby数据库里。  

执行如下命令方可  

 在hive安装目录下,执行 ./bin/schematool -initSchema -dbType derby  

 

  参考: http://fengshulin.iteye.com/blog/2320359

 

  问题五:

Exception in thread "main"java.lang.RuntimeException: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

 

解决方案:

在hive 安装目录下,创建一个临时的IO文件iotmp。比如hive的安装目录是/home/hadoop/app/hive

然后将路径配置到hive-site.xml文件的以下参数中:

< property> < name>hive.querylog.location< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Location of Hive run time structured log file< /description> < /property> < property> < name>hive.exec.local.scratchdir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Local scratch space for Hive jobs< /description> < /property> < property> < name>hive.downloaded.resources.dir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Temporary local directory for added resources in the remote file system.< /description> < /property> 问题六: Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

  解决办法:

[hadoop@HadoopSlave1 hive-1.2.1]$ hive --service metastore &

   如果实在不行,就,再参考这篇。https://my.oschina.net/AlbertHa/blog/339022

  再实在不行,见http://www.aboutyun.com/thread-11131-1-1.html 问题七:

[hadoop@HadoopSlave1 hive-1.2.1]$ bin/hive

Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 14 more
Caused by: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: hfds)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213)
at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:106)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:140)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:146)
at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:159)
at org.apache.hadoop.hive.metastore.Warehouse.getDefaultDatabasePath(Warehouse.java:177)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:600)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 19 more
[hadoop@HadoopSlave1 hive-1.2.1]$

将hadoop中share/hadoop/yarn/lib路径下jline包换成hive中匹配的jar包

 

 

 
欢迎大家,加入我的微信公众号:大数据躺过的坑     免费给分享
 
 
 

同时,大家可以关注我的个人博客

   http://www.cnblogs.com/zlslch/   和     http://www.cnblogs.com/lchzls/ 

   详情请见:http://www.cnblogs.com/zlslch/p/7473861.html

 

 

  人生苦短,我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神,汇聚于互联网和个人学习工作的精华干货知识,一切来于互联网,反馈回互联网。

  目前研究领域:大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及:Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里,每天必须有收获

 

       以及对应本平台的QQ群:161156071(大数据躺过的坑)

 

 

 

 

 

 

 

 

 

 

 

转载于:https://www.cnblogs.com/zlslch/p/5944887.html

你可能感兴趣的文章
舒适的路线
查看>>
我的2017——求职篇(四)
查看>>
0x51
查看>>
第一个spring boot 程序
查看>>
Seasar2 DI注入的底层实现
查看>>
eclipse创建maven web项目
查看>>
对象属性查找顺序
查看>>
Hnu 12505 字符串处理
查看>>
MVC - Model - Controller - View
查看>>
Android中的进程与线程
查看>>
QT之sqlite连接
查看>>
Microsoft Baseline Configuration Analyzer 2.0(MBCA) For Windows Server 2012
查看>>
SU Demos 03T-F Analysis-03Suphasevel
查看>>
继承(day09)
查看>>
javaWeb学习之tomcat服务器
查看>>
Sass入门——基本特性-基础
查看>>
c++ 一个构造函数 调用 另一个 构造函数
查看>>
Vue 仿QQ左滑删除组件
查看>>
分割线
查看>>
xls的读写
查看>>