Working in "Advanced Web Scraping App" but not in Desktop App?


I can parse the info I want with the "Advanced Web Scraping App" and download it to a CSV file no problem (I want to parse the 'quotes').

But when I do the same thing using the "Data Scraping Studio" desktop app it won't work, why?

With the "Advanced Web Scraping App" it says "tbody" in the "CSS" field and I chose "TEXT" then as csv-file, works perfect.

I tried to use that in the "Data Scraping Studio" but when I see the output file its empty?

I also tried to create a "New Agent" but it won't work. I chose input as "Direct URL" ("") and output as file csv. In "Crawl Setup" I have tried "body", "tbody" and also used "developer tool" in Chrome with the "right click" and "Copy > Copy Selector" function which gave the result "body". Did not work either.

Posted by 1 years ago

I am still pretty new to scraping but have run into the same issue, repeatedly. AFAIK The page you are looking to scrape loads its data from a .json file dynamically. When you are using the Chrome extension the page has been built and is ready to be scraped.  However the desktop app never actually "loads" a real page so those fields look like they are empty.

Luckily the .json file you are looking for is readily available and easy to read. url: [ ]

Here is a good Chrome extension for viewing .json files.

As far as getting the .json to work in the desktop app, I can not be much help as I can not get it to work either.  Here is the link to the documentation on scraping with json, however the instructions are not very clear.  Good luck! 


Posted by Andrew Soukup 1 years ago

Topic Closed! This question is closed and don't accept posts now.

Close me