我在Java 8程序中运行以下行
SparkConf sparkConf = new SparkConf();
sparkConf.setAppName("testJob");
sparkConf.setMaster("spark://blahblah:7077");
SparkSession sparkSession = SparkSession.builder().config(sparkConf).getOrCreate();
SQLContext sqlContext = new SQLContext(sparkSession);
Dataset<Row> ds = sqlContext.sql("SHOW TABLES");
我得到了一个非常神秘的异常(运行
sqlContext.sql("SHOW TABLES")行)java.lang.RuntimeException: error reading Scala signature of
org.apache.spark.sql.package: assertion failed: unsafe symbol
DeveloperApi (child of package annotation) in runtime reflection universe
(完整的堆栈跟踪记录在此问题的末尾)。
谁能给我一个关于如何解决这个问题的提示？这是什么意思？
[Gogo shell] INFO org.apache.spark.util.Utils - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 51077.
[Gogo shell] INFO org.apache.spark.network.netty.NettyBlockTransferService - Server created on 172.29.30.215:51077
[Gogo shell] INFO org.apache.spark.storage.BlockManagerMaster - Registering BlockManager BlockManagerId(driver, 172.29.30.215, 51077)
[dispatcher-event-loop-0] INFO org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint - Executor updated: app-20160921162135-0005/0 is now RUNNING
[dispatcher-event-loop-3] INFO org.apache.spark.storage.BlockManagerMasterEndpoint - Registering block manager 172.29.30.215:51077 with 2004.6 MB RAM, BlockManagerId(driver, 172.29.30.215, 51077)
[Gogo shell] INFO org.apache.spark.storage.BlockManagerMaster - Registered BlockManager BlockManagerId(driver, 172.29.30.215, 51077)
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@18e8216b{/metrics/json,null,AVAILABLE}
[Gogo shell] INFO org.apache.spark.scheduler.cluster.StandaloneSchedulerBackend - SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
[Gogo shell] WARN org.apache.spark.SparkContext - Use an existing SparkContext, some configuration may not take effect.
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@5b855aa1{/SQL,null,AVAILABLE}
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@5df316cb{/SQL/json,null,AVAILABLE}
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@78a44f1f{/SQL/execution,null,AVAILABLE}
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@7e3bbe8{/SQL/execution/json,null,AVAILABLE}
[Gogo shell] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@1476f413{/static/sql,null,AVAILABLE}
[Gogo shell] INFO org.apache.spark.sql.internal.SharedState - Warehouse path is 'file:/Users/matteorulli/git/nonono/com.nonono.osgi.example.int/com.nonono.osgi.example.int.etl/spark-warehouse'.
[Gogo shell] INFO org.apache.spark.sql.execution.SparkSqlParser - Parsing command: SHOW TABLES
java.lang.AssertionError: assertion failed: unsafe symbol DeveloperApi (child of package annotation) in runtime reflection universe
at scala.reflect.internal.Symbols$Symbol.<init>(Symbols.scala:184)
at scala.reflect.internal.Symbols$TypeSymbol.<init>(Symbols.scala:3009)
at scala.reflect.internal.Symbols$ClassSymbol.<init>(Symbols.scala:3201)
at scala.reflect.internal.Symbols$StubClassSymbol.<init>(Symbols.scala:3496)
at scala.reflect.internal.Symbols$Symbol.newStubSymbol(Symbols.scala:498)
at scala.reflect.internal.pickling.UnPickler$Scan.readExtSymbol$1(UnPickler.scala:258)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbol(UnPickler.scala:284)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolRef(UnPickler.scala:649)
at scala.reflect.internal.pickling.UnPickler$Scan.readType(UnPickler.scala:417)
at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan.at(UnPickler.scala:179)
at scala.reflect.internal.pickling.UnPickler$Scan.readTypeRef(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan.readAnnotationInfo(UnPickler.scala:492)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolAnnotation(UnPickler.scala:515)
at scala.reflect.internal.pickling.UnPickler$Scan.run(UnPickler.scala:97)
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:38)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.info(SynchronizedSymbols.scala:189)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:286)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:341)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply$mcV$sp(SymbolLoaders.scala:74)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType.complete(SymbolLoaders.scala:71)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.info(SynchronizedSymbols.scala:174)
at scala.reflect.internal.Types$TypeRef.thisInfo(Types.scala:2194)
at scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2199)
at scala.reflect.internal.tpe.FindMembers$FindMemberBase.<init>(FindMembers.scala:17)
at scala.reflect.internal.tpe.FindMembers$FindMember.<init>(FindMembers.scala:219)
at scala.reflect.internal.Types$Type.scala$reflect$internal$Types$Type$$findMemberInternal$1(Types.scala:1014)
at scala.reflect.internal.Types$Type.findMember(Types.scala:1016)
at scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:631)
at scala.reflect.internal.Types$Type.member(Types.scala:600)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:45)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:77)
at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:122)
at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:22)
at org.apache.spark.sql.types.StringType$$typecreator1$1.apply(StringType.scala:37)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:232)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:232)
at org.apache.spark.sql.types.AtomicType.<init>(AbstractDataType.scala:136)
at org.apache.spark.sql.types.StringType.<init>(StringType.scala:32)
at org.apache.spark.sql.types.StringType$.<init>(StringType.scala:48)
at org.apache.spark.sql.types.StringType$.<clinit>(StringType.scala)
at org.apache.spark.sql.execution.command.ShowTablesCommand.<init>(tables.scala:556)
at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitShowTables$1.apply(SparkSqlParser.scala:124)
at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitShowTables$1.apply(SparkSqlParser.scala:124)
at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:96)
at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitShowTables(SparkSqlParser.scala:123)
at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitShowTables(SparkSqlParser.scala:53)
at org.apache.spark.sql.catalyst.parser.SqlBaseParser$ShowTablesContext.accept(SqlBaseParser.java:844)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:42)
at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:64)
at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:64)
at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:96)
at org.apache.spark.sql.catalyst.parser.AstBuilder.visitSingleStatement(AstBuilder.scala:63)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:54)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:53)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:82)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:46)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:682)
at com.nonono.osgi.example.int.etl.command.EtlCommand.etl(EtlCommand.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.gogo.runtime.Reflective.invoke(Reflective.java:137)
at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)
at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119)
at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)
at org.apache.felix.gogo.shell.Console.run(Console.java:62)
at org.apache.felix.gogo.shell.Shell.console(Shell.java:203)
at org.apache.felix.gogo.shell.Shell.gosh(Shell.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.gogo.runtime.Reflective.invoke(Reflective.java:137)
at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)
at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119)
at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)
at org.apache.felix.gogo.shell.Activator.run(Activator.java:75)
at java.lang.Thread.run(Thread.java:745)
java.lang.ExceptionInInitializerError
at org.apache.spark.sql.execution.command.ShowTablesCommand.<init>(tables.scala:556)
at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitShowTables$1.apply(SparkSqlParser.scala:124)
at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitShowTables$1.apply(SparkSqlParser.scala:124)
at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:96)
at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitShowTables(SparkSqlParser.scala:123)
at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitShowTables(SparkSqlParser.scala:53)
at org.apache.spark.sql.catalyst.parser.SqlBaseParser$ShowTablesContext.accept(SqlBaseParser.java:844)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:42)
at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:64)
at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:64)
at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:96)
at org.apache.spark.sql.catalyst.parser.AstBuilder.visitSingleStatement(AstBuilder.scala:63)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:54)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:53)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:82)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:46)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:682)
at com.nonono.osgi.example.int.etl.command.EtlCommand.etl(EtlCommand.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.gogo.runtime.Reflective.invoke(Reflective.java:137)
at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)
at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119)
at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)
at org.apache.felix.gogo.shell.Console.run(Console.java:62)
at org.apache.felix.gogo.shell.Shell.console(Shell.java:203)
at org.apache.felix.gogo.shell.Shell.gosh(Shell.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.gogo.runtime.Reflective.invoke(Reflective.java:137)
at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)
at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119)
at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)
at org.apache.felix.gogo.shell.Activator.run(Activator.java:75)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: error reading Scala signature of org.apache.spark.sql.package: assertion failed: unsafe symbol DeveloperApi (child of package annotation) in runtime reflection universe
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.info(SynchronizedSymbols.scala:189)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:286)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:341)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply$mcV$sp(SymbolLoaders.scala:74)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType.complete(SymbolLoaders.scala:71)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.info(SynchronizedSymbols.scala:174)
at scala.reflect.internal.Types$TypeRef.thisInfo(Types.scala:2194)
at scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2199)
at scala.reflect.internal.tpe.FindMembers$FindMemberBase.<init>(FindMembers.scala:17)
at scala.reflect.internal.tpe.FindMembers$FindMember.<init>(FindMembers.scala:219)
at scala.reflect.internal.Types$Type.scala$reflect$internal$Types$Type$$findMemberInternal$1(Types.scala:1014)
at scala.reflect.internal.Types$Type.findMember(Types.scala:1016)
at scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:631)
at scala.reflect.internal.Types$Type.member(Types.scala:600)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:45)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:77)
at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:122)
at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:22)
at org.apache.spark.sql.types.StringType$$typecreator1$1.apply(StringType.scala:37)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:232)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:232)
at org.apache.spark.sql.types.AtomicType.<init>(AbstractDataType.scala:136)
at org.apache.spark.sql.types.StringType.<init>(StringType.scala:32)
at org.apache.spark.sql.types.StringType$.<init>(StringType.scala:48)
at org.apache.spark.sql.types.StringType$.<clinit>(StringType.scala)
... 49 more
最佳答案
事实证明，类路径中缺少包含DeveloperApi类的 Spark jar ，并且该类未包含在spark包中。
因此，在这种情况下，unsafe symbol DeveloperApi in runtime reflection universe转换为更平淡的ClassNotFoundException DeveloperApi:毕竟，异常消息Java接缝比scala更为简洁。
关于java - 断言: unsafe symbol DeveloperApi in runtime reflection universe，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/39620703/