hadoop Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.a

admin

[admin@hadoop-1 hadoop-3.2.2]$ hadoop jar /data/module/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar pi 10 100
+ n) D' H6 {; Y, H5 q7 j: `Number of Maps  = 10
Samples per Map = 100
Wrote input for Map #0
: Q# [5 {& ~1 SWrote input for Map #1
& \( y8 a, h) e% G, AWrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
  X9 Z. V% K" z% xWrote input for Map #6
8 g" [6 X) H: \0 n9 W, BWrote input for Map #7
7 Z! j5 K: n6 X0 u+ ?7 o& d& pWrote input for Map #8
9 `9 N/ w2 I7 |Wrote input for Map #9
Starting Job
# R3 U6 ]/ p6 e* i0 G) o2 ^2022-11-16 15:32:34,946 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
2022-11-16 15:32:35,174 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/admin/.staging/job_1668583281306_0001
9 L- ~! i. A! M$ {, |7 f+ x& o9 @2022-11-16 15:32:35,389 INFO input.FileInputFormat: Total input files to process : 10
. g  X4 `# ^6 n5 @9 @$ T2022-11-16 15:32:35,592 INFO mapreduce.JobSubmitter: number of splits:10
2022-11-16 15:32:35,806 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1668583281306_0001
0 P3 v; x( V3 r, t" ?) U" ~2022-11-16 15:32:35,808 INFO mapreduce.JobSubmitter: Executing with tokens: []
; g% y/ l; O+ }# i- D( x2022-11-16 15:32:36,044 INFO conf.Configuration: resource-types.xml not found
/ N4 a+ P; A, L) U. r2022-11-16 15:32:36,044 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
9 G- h9 }. I, p1 c5 _2022-11-16 15:32:36,423 INFO impl.YarnClientImpl: Submitted application application_1668583281306_0001
2022-11-16 15:32:36,504 INFO mapreduce.Job: The url to track the job: http://hadoop-2:8088/proxy/application_1668583281306_0001/
5 s2 J* x6 c# _9 x6 G! r2022-11-16 15:32:36,505 INFO mapreduce.Job: Running job: job_1668583281306_0001
0 u, m) ?! l1 O1 x, z) \' ~( f2022-11-16 15:32:41,561 INFO mapreduce.Job: Job job_1668583281306_0001 running in uber mode : false
7 a( X) d( L' S- z' v- S2022-11-16 15:32:41,563 INFO mapreduce.Job:  map 0% reduce 0%
, y9 L" `8 d5 Y9 I( K! F8 F2022-11-16 15:32:41,583 INFO mapreduce.Job: Job job_1668583281306_0001 failed with state FAILED due to: Application application_1668583281306_0001 failed 2 times due to AM Container for appattempt_1668583281306_0001_000002 exited with  exitCode: 1
; Q3 q8 v8 W" h& z; T! kFailing this attempt.Diagnostics: [2022-11-16 15:32:40.765]Exception from container-launch.
' q, j' \  |: B3 R- I& Y: S1 s" |' t. |Container id: container_1668583281306_0001_02_000001
Exit code: 1
% V6 A0 W" ]) L9 `6 `" E
+ h" F6 O  \8 t( z2 U5 O[2022-11-16 15:32:40.799]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
8 I3 f3 n  I5 CLast 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
; A/ Q! r" ]5 r& p" f1 \Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

) F* G4 v7 l, s7 t$ O0 mPlease check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
! h) w2 z' K/ a- l4 G& X7 _. i+ G  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
) ]8 D  u, j& A( O</property>
; K  m: p& X: R8 m<property>
0 @) L' M+ u9 a$ J* u: y  <name>mapreduce.map.env</name>
6 |; {' z# T& S) N# C+ h5 p. [  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
) o8 J0 m3 s) q- _' X) B' q</property>
! [( ]) k$ I- e<property>
  <name>mapreduce.reduce.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
, S/ \3 h" }9 R4 g7 `' f3 [6 M</property>
1 ^: I* v1 _1 k; L- r4 ^' e
[2022-11-16 15:32:40.800]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
: d7 v" `9 P& q3 A$ ^Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
( G% V6 f/ i2 V. m/ ^Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
* y  J2 y) `% M0 D
! \( s/ _4 m! ?$ G" k$ c4 O2 vPlease check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
6 W5 h7 `& A+ k. o<property>
2 Q& b: f( S3 H0 N, O& v% j  <name>mapreduce.map.env</name>
! ~8 Z6 p; @7 _3 Y, F  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.reduce.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>

For more detailed output, check the application tracking page: http://hadoop-2:8088/cluster/app/application_1668583281306_0001 Then click on links to logs of each attempt.
1 N6 f4 f4 R% V# p3 U) b3 u. Failing the application.
' U  E- u' g* L2022-11-16 15:32:41,608 INFO mapreduce.Job: Counters: 0
+ `7 K7 c- J# c4 U! H. u6 H. `, FJob job_1668583281306_0001 failed!

分析失败，解决方法：

; t8 U6 @9 f1 q8 p[admin@hadoop-1 hadoop-3.2.2]$ vim etc/hadoop/mapred-site.xml
添加如下内容：
' x, p, g% h6 K! ^
<property>
  <name>yarn.app.mapreduce.am.env</name>
: I5 @, d# d( X  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
& x$ ?# _0 Y* B4 d4 L<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
" j: v7 l8 M# l5 I& ]: ?5 J* t% E<property>
; T. S+ z) x  u. w0 B3 r  <name>mapreduce.reduce.env</name>
4 M5 Z' v" i. @2 t: K  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>


$ V7 f- u+ W8 b' {. x同步到其他节点：



2 `0 z& L" ^3 f4 r
[admin@hadoop-1 hadoop-3.2.2]$ xsync etc/hadoop/
) g% u1 F! s8 f& \==============hadoop-1==================
sending incremental file list
1 ?4 n' C0 [/ n  z: _" j
sent 1,004 bytes  received 18 bytes  2,044.00 bytes/sec
total size is 115,250  speedup is 112.77
  ]. `  ^3 G/ b+ r==============hadoop-2==================
/ h, {# w+ ~4 p4 N- c3 v7 osending incremental file list
3 @# z3 o. _4 H$ V* j- hhadoop/
hadoop/mapred-site.xml

& x! F1 N, f( ~  v, @2 Lsent 1,251 bytes  received 52 bytes  2,606.00 bytes/sec
  W  J* L5 S* [- }1 mtotal size is 115,250  speedup is 88.45
' m, A8 g7 F7 S==============hadoop-3==================
( ?" m1 {% A1 g/ b" e+ nsending incremental file list
hadoop/
% [) I% i1 `2 u' E' C$ k( yhadoop/mapred-site.xml
/ u3 {0 S; k& c. Y
sent 1,251 bytes  received 52 bytes  2,606.00 bytes/sec
total size is 115,250  speedup is 88.45

. u0 D% w$ u# j4 Q  C
5 w" d% p9 T7 o* {/ O* T再次验证：

[admin@hadoop-1 hadoop-3.2.2]$ hadoop jar /data/module/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar pi 10 100
6 w/ Z  g, ^! l# `' tNumber of Maps  = 10
Samples per Map = 100
Wrote input for Map #0
1 u! R) U3 w: u7 ~/ _8 R$ ]Wrote input for Map #1
; {: v7 ?$ w2 y2 g! ]" UWrote input for Map #2
Wrote input for Map #3
! \+ R2 u2 _# f) {Wrote input for Map #4
! {  V" d8 H9 jWrote input for Map #5
Wrote input for Map #6
( Z' v) ?( z8 q! Y  dWrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
8 ?7 x7 I* S: }8 p+ @2022-11-16 15:36:05,630 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
2022-11-16 15:36:05,878 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/admin/.staging/job_1668583281306_0002
* ]* D6 X  e8 g2 |- u4 F( v2022-11-16 15:36:06,065 INFO input.FileInputFormat: Total input files to process : 10
2022-11-16 15:36:06,214 INFO mapreduce.JobSubmitter: number of splits:10
$ s  z: M3 {- e, a! G4 p8 B2022-11-16 15:36:06,444 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1668583281306_0002
! A" W3 e+ Z3 s; D2022-11-16 15:36:06,446 INFO mapreduce.JobSubmitter: Executing with tokens: []
2022-11-16 15:36:06,654 INFO conf.Configuration: resource-types.xml not found
2022-11-16 15:36:06,654 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-11-16 15:36:06,928 INFO impl.YarnClientImpl: Submitted application application_1668583281306_0002
, E$ Q# J1 m( G2022-11-16 15:36:06,981 INFO mapreduce.Job: The url to track the job: http://hadoop-2:8088/proxy/application_1668583281306_0002/
2022-11-16 15:36:06,982 INFO mapreduce.Job: Running job: job_1668583281306_0002
4 s0 K& X  H4 G0 r* \8 u. J  u( R2022-11-16 15:36:17,134 INFO mapreduce.Job: Job job_1668583281306_0002 running in uber mode : false
( Y% k7 `9 C1 V2022-11-16 15:36:17,136 INFO mapreduce.Job:  map 0% reduce 0%
2022-11-16 15:36:23,297 INFO mapreduce.Job:  map 40% reduce 0%
3 _. s4 Q3 \2 k2 M  @5 M% O- H2022-11-16 15:36:24,308 INFO mapreduce.Job:  map 100% reduce 0%
3 \! Z6 Q, P+ k2 w& A  B' d2022-11-16 15:36:28,342 INFO mapreduce.Job:  map 100% reduce 100%
+ E9 e; n& o% R# z3 G2022-11-16 15:36:29,360 INFO mapreduce.Job: Job job_1668583281306_0002 completed successfully
+ l. R$ ^0 `- s8 N' u$ {8 P( R5 e2022-11-16 15:36:29,504 INFO mapreduce.Job: Counters: 54
+ D' h  w4 f3 z' B, B        File System Counters
                FILE: Number of bytes read=226
                FILE: Number of bytes written=2630034
: k5 C. N" @% c; H3 N/ ]                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=2600
                HDFS: Number of bytes written=215
* s! Q1 \4 P# {0 l: R+ k! Q                HDFS: Number of read operations=45
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=3
: v% W" \/ O! Q# s8 V% W                HDFS: Number of bytes read erasure-coded=0
0 ]) z' R5 A. M        Job Counters
0 o8 M& |  g; b5 O                Launched map tasks=10
- f$ B$ b3 p; J5 J4 x* p3 h                Launched reduce tasks=1
                Data-local map tasks=10
                Total time spent by all maps in occupied slots (ms)=41394
# k; N0 z4 W1 a$ \; U: }                Total time spent by all reduces in occupied slots (ms)=3116
                Total time spent by all map tasks (ms)=41394
/ W# U% [: Q' e  s0 J  \3 m7 V  C                Total time spent by all reduce tasks (ms)=3116
# {7 o9 Z1 e/ m* Z! S7 S  X& C. v                Total vcore-milliseconds taken by all map tasks=41394
9 a1 @- U2 C8 t) u+ }                Total vcore-milliseconds taken by all reduce tasks=3116
                Total megabyte-milliseconds taken by all map tasks=42387456
                Total megabyte-milliseconds taken by all reduce tasks=3190784
5 F0 N% y) V* {; q) o0 k& T        Map-Reduce Framework
                Map input records=10
$ K6 T$ m5 Z8 [/ N                Map output records=20
( D$ C/ |+ `4 h2 U& Q                Map output bytes=180
                Map output materialized bytes=280
                Input split bytes=1420
  m1 P+ v0 t% U# |8 I1 M. O0 b# C                Combine input records=0
                Combine output records=0
                Reduce input groups=2
3 `1 x& C, y9 }! C8 ^6 m                Reduce shuffle bytes=280
3 M( k! t# J& s; Y& M1 o                Reduce input records=20
3 U" \4 l9 X9 k( K) p                Reduce output records=0
                Spilled Records=40
& F4 j) g( t' @8 X' C% d                Shuffled Maps =10
                Failed Shuffles=0
8 e  g& c8 c) p; `$ U5 i                Merged Map outputs=10
                GC time elapsed (ms)=1915
                CPU time spent (ms)=6790
                Physical memory (bytes) snapshot=3809210368
/ o+ h* t( k3 w# G: s4 C; U                Virtual memory (bytes) snapshot=31150141440
                Total committed heap usage (bytes)=3503292416
                Peak Map Physical memory (bytes)=360734720
                Peak Map Virtual memory (bytes)=2835021824
; I2 w$ j! e4 e; l" y1 r( r                Peak Reduce Physical memory (bytes)=279707648
/ D# Z1 K& l( N8 z4 a" O5 q                Peak Reduce Virtual memory (bytes)=2839097344
        Shuffle Errors
0 E3 C$ M8 ^% i7 e  b                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=1180
        File Output Format Counters
                Bytes Written=97
3 B6 Y& L# q/ q" B2 _1 V3 D6 J5 JJob Finished in 24.227 seconds
Estimated value of Pi is 3.14800000000000000000

7 e5 k4 ?- _6 l问题解决
4 c' F! v( D; ]! `9 s