作者:JesseZhang (CNZQS|JesseZhang)
博客地址:http://www.cnzqs.com
在创建自定义的Mapper时候,编译正确,但上传到集群执行时出现错误:
11/12/11 22:53:16 INFO mapred.JobClient: Task Id : attempt_201111301626_0015_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: actiondemo.MyJob$MapClass
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: actiondemo.MyJob$MapClass
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
... 8 more
经过验证,发现问题原因及解决办法如下:
因为使用的是0.20以上的Hadoop版本,在调用jar中的自定义mapper时,需要设置setJarByClass方法,设置方法如下:
job.setJarByClass(MyJob.class);
其实,在输出日志中也有提示信息:
11/12/11 22:53:03 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
job.setJarByClass(MyJob.class);
-----------------------------------------
分享到:
相关推荐
解决方案:Exceptionin thread "main" java.lang.UnsatisfiedLinkError:org.apache.hadoop.util.NativeCrc32.nativeCo
【SpringBoot】Error: java.lang.ClassNotFoundException: org.wltea.analyzer.core.IKSegmenter报错明细问题分析后记 报错明细 IDEA SpringBoot集成hadoop运行环境,本地启动项目,GET请求接口触发远程提交...
报错:java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.-附件资源
【kettle集成cdh6.1】外部数据源读写hdfs若干错前言环境报错问题分析解决办法后记 前言 最近试着上手了一下kettle,搭建过程很简单,就是下载个包解压一下,但是在配置数据源的过程中着实踩了不少坑,这里记录一下。...
我在进行MapReduce本地测试的时候,遇见下面的报错: ...解决方法:①第一步:在项目工程中加入NativelO.java,注意包名必须与源码中一致 ②第二步:将access方法的返回值改为true,不让他去调用access0() 类似
Exception in thread "main" java.lang.UnsatisfiedLinkError:''boolean org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(java.lang.String, int)' * at org.apache.hadoop.io.nativeio.NativeIO$...
运行Sqoop报错:Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/LogicalType,下载此资源放到Sqoop的lib目录下即可
at org.apache.hadoop.util.Shell.runCommand(Shell.java:482) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at ...
org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected at org.apache...
[Hadoop权威指南(第2版)].(Hadoop:The.Definitive.Guide).Tom.White.文字版.pdf
解决运行Spark、Hadoop程序出现:Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createFileWithMode0(Ljava/lang/String;JJJI)Ljava/io/FileDescriptor;问题
win10环境下编译的hadoop-2.9.2的资源:hadoop.dll, wintools.exe
at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:45) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:163) at org...
报错:java.lang.Exception: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z 所需文件
解决windows下远程调试hadoop权限问题( ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator...1.2.1,注释了包中org.apache.hadoop.fs下FileUtil.class中的checkReturnValue的验证方法
【SpringBoot】Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster报错明细问题解决后记 报错明细 IDEA SpringBoot集成hadoop运行环境,,本地启动项目,GET请求接口触发...
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 通过控制台的...
2)Caused by: java.lang.OutOfMemoryError: Java heap space 3)Current usage: 1.0 GB of 1 GB physical memory used;2.7 GB of 2.1 GB virtual memory used. Killing container 4)java.lang.RuntimeException:...