Q : 在将RDD计算结果写入MySQL中Spark报错,如下所示:
java.lang.UnsupportedClassVersionError: com/mysql/jdbc/Driver : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190)相关部分代码:
rdd.foreachPartition(iter => { var conn: Connection = null var ps: PreparedStatement = null val tableName = "SOME_TABLE" val sql = s"INSERT INTO $tableName(value1,value2,value3,value4) VALUES(?,?,?,?)" try { Class.forName("com.mysql.jdbc.Driver").newInstance conn = DriverManager.getConnection("jdbc:MYSQSL://HOST.DB:8868/DB", "USERNAME", "PASSWORD") while (iter.hasNext) { val msg = iter.next() ps = conn.prepareStatement(sql) val value1 = timestamp val value2 = msg._1._2 val value3 = msg._1._1 val value4 = msg._2 ps.setTimestamp(1, java.sql.Timestamp.valueOf(value1)) ps.setString(2, value2) ps.setString(3, value3) ps.setInt(4, value4) ps.executeUpdate() } } catch { case e: Exception => println("-> Exception!\t:\t" + e) } finally { if (ps != null) ps.close() if (conn != null) conn.close() } })A: 我的解决方法 : 使用特定版本的mysql-connector-java 尝试过使用6.0.6和5.1.14两个版本, 后来发现只有与spark中版本一致的5.1.38可以正常使用.
<dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.38</version> </dependency>网上给出的原因更多的指向编译时和运行时的JDK版本不同导致,例如使用JDK 8.0进行编译,却在JDK7.0的JVM上进行运行,则往往会这样的错误. 这和我遇到的情况不同, 但可以作为参考.
附上更为详尽解释原因的链接:
How to fix java.lang.UnsupportedClassVersionError: Unsupported major.minor version
java.lang.UnsupportedClassVersionError: Unsupported major.minor version 51.0的错误
Peculiar Eclipse java.lang.UnsupportedClassVersionError, Unsupported major.minor version 52.0
