hadoop Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.a

admin

[admin@hadoop-1 hadoop-3.2.2]$ hadoop jar /data/module/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar pi 10 100
Number of Maps  = 10
/ i# x& Q+ b! Y& m+ FSamples per Map = 100
9 u+ Q& T4 v4 d! u9 _; HWrote input for Map #0
Wrote input for Map #1
5 `& r4 z. Z  `4 k/ D$ ZWrote input for Map #2
/ t% r1 \7 y( [Wrote input for Map #3
3 Z  m) F# v6 I  u0 R7 gWrote input for Map #4
1 G& U% A7 t' `Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
; s( w8 v, r( F' l2022-11-16 15:32:34,946 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
$ d8 z+ g: ~7 ]1 h1 v) i- C: j: v2022-11-16 15:32:35,174 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/admin/.staging/job_1668583281306_0001
2022-11-16 15:32:35,389 INFO input.FileInputFormat: Total input files to process : 10
6 q. e1 p# e& ^3 h2022-11-16 15:32:35,592 INFO mapreduce.JobSubmitter: number of splits:10
2022-11-16 15:32:35,806 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1668583281306_0001
, D: R* M+ E& K0 A* U2 R0 i  v2022-11-16 15:32:35,808 INFO mapreduce.JobSubmitter: Executing with tokens: []
1 U7 X& _3 y+ y8 r* W; {4 }2022-11-16 15:32:36,044 INFO conf.Configuration: resource-types.xml not found
- A; g% U( ~) A. I& v' f, u7 j6 W2022-11-16 15:32:36,044 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
0 E! P  r) t) Q/ d% Q: M2022-11-16 15:32:36,423 INFO impl.YarnClientImpl: Submitted application application_1668583281306_0001
2022-11-16 15:32:36,504 INFO mapreduce.Job: The url to track the job: http://hadoop-2:8088/proxy/application_1668583281306_0001/
3 D# ]' w5 h8 p4 K& [9 w) [2 Z2022-11-16 15:32:36,505 INFO mapreduce.Job: Running job: job_1668583281306_0001
2022-11-16 15:32:41,561 INFO mapreduce.Job: Job job_1668583281306_0001 running in uber mode : false
8 r. B' g- Y4 }0 @+ ~1 X$ B2022-11-16 15:32:41,563 INFO mapreduce.Job:  map 0% reduce 0%
1 m  Q. t9 y, d3 a& n, x2 l7 }5 ]2022-11-16 15:32:41,583 INFO mapreduce.Job: Job job_1668583281306_0001 failed with state FAILED due to: Application application_1668583281306_0001 failed 2 times due to AM Container for appattempt_1668583281306_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2022-11-16 15:32:40.765]Exception from container-launch.
Container id: container_1668583281306_0001_02_000001
3 h% z! V2 H! u7 {9 P- b3 MExit code: 1
7 e3 ?3 \; K* w4 X8 c. Z* t' n5 `3 @
% A( |* M- V+ r& Q& b8 F[2022-11-16 15:32:40.799]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
- S# |1 F$ D+ p$ e5 j6 n  f. j<property>
8 o: j/ e7 ?% A7 F  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
# ~8 u. N; ]" R+ r</property>
( G% Z1 S! H/ f  X: ~<property>
  <name>mapreduce.reduce.env</name>
' A, V4 ^1 W6 P; B* \' t  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
/ |& i7 \: I1 i7 }1 Z</property>
5 A- Z0 I9 P$ s1 t7 U6 L! g
[2022-11-16 15:32:40.800]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
3 y* Y  }$ F; n# i  f% L% ELast 4096 bytes of prelaunch.err :
- u* T( v: E; C! QLast 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
& k4 V0 j- F2 B; O8 s9 [8 Q' ]' z  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
, u; s# `# b6 ^5 n  S- D) {, }2 d</property>
<property>
% Q6 Q4 R+ h+ z) j/ O! Z$ P4 n  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
. I$ d3 Z, w' D1 ]7 G$ x; n<property>
6 ~5 w3 S' k3 m& V& ]  <name>mapreduce.reduce.env</name>
3 ^& E# m0 Q6 F- d" L  i- _  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
1 e1 j) T8 i* e' K- l: \' `0 ^# f9 N
9 Z5 w/ H4 n: x. m) yFor more detailed output, check the application tracking page: http://hadoop-2:8088/cluster/app/application_1668583281306_0001 Then click on links to logs of each attempt.
. Failing the application.
) p; Y, F  U8 {2022-11-16 15:32:41,608 INFO mapreduce.Job: Counters: 0
Job job_1668583281306_0001 failed!
6 r0 j0 n5 U) L5 \" S* R
. l0 m3 ^$ T, V' N! e, ~4 z+ c分析失败，解决方法：
" I" H( b4 w$ W
[admin@hadoop-1 hadoop-3.2.2]$ vim etc/hadoop/mapred-site.xml
添加如下内容：

<property>
3 }3 F- F! T- _7 r& J/ D  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
: ], N+ z, g( K) y( Z</property>
<property>
  <name>mapreduce.map.env</name>
* b/ u/ O/ F. v/ V+ v( R+ W, s  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
<property>
5 k# h3 |1 y( ^  <name>mapreduce.reduce.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
3 g' c; q$ v0 ^! x</property>
4 [& h- Q" B/ A' L
  m# P8 P: K2 J# X" n/ D; I, b
同步到其他节点：

$ y  U' j) y; S6 w5 o1 u8 Q

( n; |$ \5 t# z/ f* ^: j
6 b5 J4 ?% g; z9 l$ `[admin@hadoop-1 hadoop-3.2.2]$ xsync etc/hadoop/
) b+ S, c1 F9 m==============hadoop-1==================
1 C! ^$ b# g! m9 s& Wsending incremental file list

sent 1,004 bytes  received 18 bytes  2,044.00 bytes/sec
( |; k) j4 Y8 H5 B- B1 l5 btotal size is 115,250  speedup is 112.77
==============hadoop-2==================
sending incremental file list
8 t, |7 f! ?% W0 U! K1 b- zhadoop/
hadoop/mapred-site.xml
* a6 l5 C- m( N# q, {  l6 H2 F5 _. o0 X
sent 1,251 bytes  received 52 bytes  2,606.00 bytes/sec
total size is 115,250  speedup is 88.45
5 ^8 l; x( `% Q. m7 \* ]% I0 r==============hadoop-3==================
- E- W5 G! ~$ n$ a1 }sending incremental file list
9 z1 `4 M+ t' @7 A" Uhadoop/
hadoop/mapred-site.xml

0 B6 H0 X( x: ]# s4 _7 x4 asent 1,251 bytes  received 52 bytes  2,606.00 bytes/sec
total size is 115,250  speedup is 88.45

+ e1 E8 |8 F, R% I1 n( w
- z; _! r' L; q0 ?( T8 o再次验证：
* ]& n5 [6 l5 p9 w3 T. t
6 j0 Y+ N' L  C& q, p[admin@hadoop-1 hadoop-3.2.2]$ hadoop jar /data/module/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar pi 10 100
+ t1 `: \- n9 w. hNumber of Maps  = 10
Samples per Map = 100
5 I/ F# _7 T$ }/ m& o+ Y6 l& |Wrote input for Map #0
5 V+ i2 C. u, `3 _$ J5 |' s8 wWrote input for Map #1
+ m" e" B' C/ T3 y* UWrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
+ D8 x) n# j4 ^; h4 L; b0 k5 xWrote input for Map #5
+ d' B9 H6 ?" {6 Y) IWrote input for Map #6
) K4 ?* t# c; `4 G0 iWrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
- ?. o" U/ h1 vStarting Job
2022-11-16 15:36:05,630 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
2 R) G; v7 y, f2 }2 Z* \' i2022-11-16 15:36:05,878 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/admin/.staging/job_1668583281306_0002
8 E4 h( E1 z7 ?7 _8 Y2022-11-16 15:36:06,065 INFO input.FileInputFormat: Total input files to process : 10
9 H* ?9 ?% F1 l  ?: O+ g2022-11-16 15:36:06,214 INFO mapreduce.JobSubmitter: number of splits:10
0 N% ?, S8 l% d- l- U4 d7 k7 E0 S4 b2022-11-16 15:36:06,444 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1668583281306_0002
0 l( R  A9 W9 I- ~2022-11-16 15:36:06,446 INFO mapreduce.JobSubmitter: Executing with tokens: []
8 g: ~' {6 F- N8 B1 x/ r3 |0 a1 R2022-11-16 15:36:06,654 INFO conf.Configuration: resource-types.xml not found
3 j; h5 Y4 f# T1 _9 y5 d9 |, u2022-11-16 15:36:06,654 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-11-16 15:36:06,928 INFO impl.YarnClientImpl: Submitted application application_1668583281306_0002
2022-11-16 15:36:06,981 INFO mapreduce.Job: The url to track the job: http://hadoop-2:8088/proxy/application_1668583281306_0002/
2 M- W. E" ^, W: u6 l& m2022-11-16 15:36:06,982 INFO mapreduce.Job: Running job: job_1668583281306_0002
9 n( U2 |0 I$ J8 a2022-11-16 15:36:17,134 INFO mapreduce.Job: Job job_1668583281306_0002 running in uber mode : false
2022-11-16 15:36:17,136 INFO mapreduce.Job:  map 0% reduce 0%
2022-11-16 15:36:23,297 INFO mapreduce.Job:  map 40% reduce 0%
1 V( C- S* l4 O1 d9 T6 ?2022-11-16 15:36:24,308 INFO mapreduce.Job:  map 100% reduce 0%
; R2 o; ]8 ?8 x+ J9 Q. N& H2022-11-16 15:36:28,342 INFO mapreduce.Job:  map 100% reduce 100%
2022-11-16 15:36:29,360 INFO mapreduce.Job: Job job_1668583281306_0002 completed successfully
2022-11-16 15:36:29,504 INFO mapreduce.Job: Counters: 54
        File System Counters
5 ?# `" w) e' Z- K: x& \5 G$ D                FILE: Number of bytes read=226
" H8 X) z/ l! a6 E) M+ V                FILE: Number of bytes written=2630034
* q% c! G. t+ H) \( V                FILE: Number of read operations=0
. h1 m$ g! ~* r2 |) O; W# h                FILE: Number of large read operations=0
  a) Y* x# \+ T# D* [                FILE: Number of write operations=0
                HDFS: Number of bytes read=2600
2 o- ^$ n& k5 \8 `: w                HDFS: Number of bytes written=215
7 G9 Z0 M) u  ^  c- W$ u+ u                HDFS: Number of read operations=45
% _7 i$ G6 P+ T: B2 k9 u6 n0 y                HDFS: Number of large read operations=0
                HDFS: Number of write operations=3
                HDFS: Number of bytes read erasure-coded=0
        Job Counters
1 Z  R! M% y. b" q$ O9 O' \8 @                Launched map tasks=10
                Launched reduce tasks=1
                Data-local map tasks=10
                Total time spent by all maps in occupied slots (ms)=41394
                Total time spent by all reduces in occupied slots (ms)=3116
- ]0 m9 ~& I& o0 X/ Y# X2 T                Total time spent by all map tasks (ms)=41394
+ I) _5 m1 d& X& [1 e$ x                Total time spent by all reduce tasks (ms)=3116
                Total vcore-milliseconds taken by all map tasks=41394
4 d( i9 l& c+ c! a* ]* W* ^                Total vcore-milliseconds taken by all reduce tasks=3116
                Total megabyte-milliseconds taken by all map tasks=42387456
( r( b4 w1 G# e                Total megabyte-milliseconds taken by all reduce tasks=3190784
        Map-Reduce Framework
: D" W: Q* E1 U1 F, N8 q3 m                Map input records=10
7 [9 x2 g. J' M' x* Z" s: M                Map output records=20
                Map output bytes=180
' _" T: [5 }  ]' v; v8 a                Map output materialized bytes=280
: L: C! X( y: f( ~" g                Input split bytes=1420
                Combine input records=0
                Combine output records=0
                Reduce input groups=2
6 D0 D3 K5 ?# a# ]8 f" y                Reduce shuffle bytes=280
6 \9 u5 D4 o4 E$ d, _( k                Reduce input records=20
! R  H0 @% A0 y. ^: a- I" @                Reduce output records=0
                Spilled Records=40
                Shuffled Maps =10
                Failed Shuffles=0
                Merged Map outputs=10
                GC time elapsed (ms)=1915
% |% b9 ^7 i4 A( f5 q, S4 i                CPU time spent (ms)=6790
: h4 y; N, A, U                Physical memory (bytes) snapshot=3809210368
, e) Q/ V3 c+ g5 h. I& Y1 Q* Y  N- V                Virtual memory (bytes) snapshot=31150141440
. v" S7 h" A& _2 _# r3 J9 ?* d                Total committed heap usage (bytes)=3503292416
+ G# o0 Z& Z9 K% L                Peak Map Physical memory (bytes)=360734720
6 t: @! W# G! Y6 i8 b+ C                Peak Map Virtual memory (bytes)=2835021824
                Peak Reduce Physical memory (bytes)=279707648
+ y& Q* O+ k$ D9 ]( L2 Y                Peak Reduce Virtual memory (bytes)=2839097344
/ {" {1 h: P0 A2 ~4 i        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
9 R' I# H. @* E5 t                WRONG_MAP=0
6 Q5 F5 K" Z* \( W                WRONG_REDUCE=0
        File Input Format Counters
- g1 V0 ?* l0 }                Bytes Read=1180
! R: V& B$ v$ d% D$ C  d0 d1 l$ S, l        File Output Format Counters
4 p8 K, J% E# e( K                Bytes Written=97
Job Finished in 24.227 seconds
Estimated value of Pi is 3.14800000000000000000
0 r4 n( _$ F, G& T' l( F' Q% w
& }3 s, w. v$ }4 C" L问题解决
* l' a/ ^+ K! `  w. |. b- `8 Z