1.spark structed streaming 报如图下面的错误

在pom.xml加这个配置即可

     
      2.3.0
      
     
      2.11
      
       
      
       org.apache.spark
        
      
       spark-sql-kafka-0-10_${scala.version}
        
      
       ${spark.version}
       
     

或在maven官网搜:spark－sql－kafka  也可以

第二个问题是运行structed streaming就无缘无故报空指针异常，看下报错日志

18/09/20 18:59:46 ERROR StreamMetadata: Error writing stream metadata StreamMetadata(ea90df83-a9ab-4e23-b03f-90bc3f8581ca) to file:/C:/Users/iwen/AppData/Local/Temp/temporary-c0117371-5f2e-419b-b45c-1dfd7609c644/metadatajava.lang.NullPointerException    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)    at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)......Exception in thread "main" java.lang.NullPointerException

解决方法

下载这个，放到这个位置(https://github.com/srccodes/hadoop-common-2.2.0-bin)

代码最上面加一句这个即可

System.setProperty("hadoop.home.dir", "C:\\hadoop-common-2.2.0")