java - Can't access sqlite db from Spark -

-

i have following code:

val conf = new sparkconf().setappname("spark test") val sc = new sparkcontext(conf) val sqlcontext = new org.apache.spark.sql.sqlcontext(sc)  val data = sqlcontext.read.format("jdbc").options(   map(     "url" -> "jdbc:sqlite:/nv/pricing/ix_tri_pi.sqlite3",     "dbtable" -> "select security_id ix_tri_pi")).load()  data.foreach {   row => println(row.getint(1)) }

and try submit with:

spark-submit \   --class "com.novus.analytics.spark.sparktest" \    --master "local[4]" \  /users/smabie/workspace/analytics/analytics-spark/target/scala-2.10/analytics-spark.jar \   --conf spark.executer.extraclasspath=sqlite-jdbc-3.8.7.jar \   --conf  spark.driver.extraclasspath=sqlite-jdbc-3.8.7.jar \   --driver-class-path sqlite-jdbc-3.8.7.jar \   --jars sqlite-jdbc-3.8.7.jar

but following exception:

exception in thread "main" java.sql.sqlexception: no suitable driver

i using spark version 1.6.1, if helps. thanks!

try defining jar last parameter of spark-submit.


Comments

Post a Comment

Popular posts from this blog

Django REST Framework perform_create: You cannot call `.save()` after accessing `serializer.data` -

-
i have following viewset: class artistprofileviewset(viewsets.modelviewset): queryset = artistprofile.objects.all() serializer_class = artistprofileserializer def perform_create(self, serializer): if serializer.is_valid(): serializer.save() with following serializers: class simpleartisttrackserializer(serializers.hyperlinkedmodelserializer): class meta: model = artisttrack fields = (...my fields...) class artistprofileserializer(serializers.hyperlinkedmodelserializer): owners = userserializer(many=true, required=false) tracks = simpleartisttrackserializer(many=true, required=false) class meta: model = artistprofile fields = (...my fields...) i getting following error: assertionerror: cannot call `.save()` after accessing `serializer.data`.if need access data before committing database inspect 'serializer.validated_data' instead. i don't see editing serializer.data object. th...
Read more

Why does Go error when trying to marshal this JSON? -

-
i'm trying rather simple example of trying decode json file structs using golang. however, when attempting error that json: cannot unmarshal object go value of type []main.track which don't understand. here's code in question, , json can found here . package main import ( "encoding/json" "fmt" "net/http" "os" ) // tracks group of track types type tracks struct { tracks []track } // track represents singular track type track struct { name string url string artist artist } // artist represents single artist type artist struct { name string mbid string url string } func main() { url := "http://ws.audioscrobbler.com/2.0/?method=geo.gettoptracks&api_key=c1572082105bd40d247836b5c1819623&format=json&country=netherlands" response, err := http.get(url) if err != nil { fmt.printf("%s\n", err) os.exit(1) } de...
Read more