Java JSoup error fetching URL -

-

i'm creating application enable me fetch values specific website console. value <span> element , i'm using jsoup.

my challenge has error:

error fetching url

here java code:

public class testsl {     public static void main(string[] args) throws ioexception {         document doc = jsoup.connect("https://stackoverflow.com/questions/11970938/java-html-parser-to-extract-specific-data").get();         elements spans = doc.select("span[class=hidden-text]");         (element span: spans) {             system.out.println(span.text());         }     } }

and here error on console:

exception in thread "main" org.jsoup.httpstatusexception: http error fetching url. status=403, url=java html parser extract specific data? @ org.jsoup.helper.httpconnection$response.execute(httpconnection.java:590) @ org.jsoup.helper.httpconnection$response.execute(httpconnection.java:540) @ org.jsoup.helper.httpconnection.execute(httpconnection.java:227) @ org.jsoup.helper.httpconnection.get(httpconnection.java:216) @ testsl.main(testsl.java:19)

what doing wrong , how can resolve it?

set user-agent header:

.useragent("mozilla")

example:

document document = jsoup.connect("https://stackoverflow.com/questions/11970938/java-html-parser-to-extract-specific-data").useragent("mozilla").get(); elements elements = document.select("span.hidden-text"); (element element : elements) {   system.out.println(element.text()); }

stack exchange

inbox

reputation , badges

source: https://stackoverflow.com/a/7523425/1048340

perhaps related: https://meta.stackexchange.com/questions/277369/a-terms-of-service-update-restricting-companies-that-scrape-your-profile-informa


Comments

Post a Comment

Popular posts from this blog

Django REST Framework perform_create: You cannot call `.save()` after accessing `serializer.data` -

-
i have following viewset: class artistprofileviewset(viewsets.modelviewset): queryset = artistprofile.objects.all() serializer_class = artistprofileserializer def perform_create(self, serializer): if serializer.is_valid(): serializer.save() with following serializers: class simpleartisttrackserializer(serializers.hyperlinkedmodelserializer): class meta: model = artisttrack fields = (...my fields...) class artistprofileserializer(serializers.hyperlinkedmodelserializer): owners = userserializer(many=true, required=false) tracks = simpleartisttrackserializer(many=true, required=false) class meta: model = artistprofile fields = (...my fields...) i getting following error: assertionerror: cannot call `.save()` after accessing `serializer.data`.if need access data before committing database inspect 'serializer.validated_data' instead. i don't see editing serializer.data object. th...
Read more

Why does Go error when trying to marshal this JSON? -

-
i'm trying rather simple example of trying decode json file structs using golang. however, when attempting error that json: cannot unmarshal object go value of type []main.track which don't understand. here's code in question, , json can found here . package main import ( "encoding/json" "fmt" "net/http" "os" ) // tracks group of track types type tracks struct { tracks []track } // track represents singular track type track struct { name string url string artist artist } // artist represents single artist type artist struct { name string mbid string url string } func main() { url := "http://ws.audioscrobbler.com/2.0/?method=geo.gettoptracks&api_key=c1572082105bd40d247836b5c1819623&format=json&country=netherlands" response, err := http.get(url) if err != nil { fmt.printf("%s\n", err) os.exit(1) } de...
Read more