I am trying to scrape free USA, UK proxies from http://www.gatherproxy.com/proxylist/country/?c=United%20States
I just need the following fields every minute accessible via an API
How can I do this using Agenty?
You may setup a scraping agent for this website to scrape the proxies. This is a HTML table, so the best selector will be
td:nth-child(n) . And then replace the
n with the column position for each field you want to scrape out.
For example, to extract the proxy_address. The selector will be :
To scrape the proxy port number, it will be
td:nth-child(3) and so on for other fields you mentioned and then save your scraping agent
Then you can schedule the agent to run every minute or via the API and pull the result in JSON using the result API as in documentation here - https://www.agenty.com/docs/api/jobs#Get_job_result_by_job_id