The List feature in Agenty is used to upload CSV, TSV or other delimited files, which allow us to create and manage large numbers of input/URLs in agents input.
Lists feature is very helpful, especially when you are scraping a large website while having their URL list with multiple columns to associate some additional data as well in agent result with the URL. Also, we cannot enter a lot of URLs in manual input text area on agent page, which might freeze your browser due to size of in-memory text.
For example, if you have a scraping agent to extract price from a large eCommerce website using the URLs. You can bulk upload the URLs using a CSV file and then change the input type to "URL LIST" and select the URL field to point your scraping agent to that uploaded URL list.
Create a List
- Go to the Lists menu from the left sidebar,
- Now click on NEW LIST to create a new list,
- A dialog box will open, where you can give a
nameto your list and can also add the
- Save it by clicking on Add button in green colour,
Agenty will re-direct to next step page where we can upload our delimited files. For example: CSV, TSV etc.
Drag and drop the file or click anywhere on drag area to upload manually,
Select the Has headers? switch if your file has the headers or un-check if no headers, Agenty will auto-generate the heading with names like
Text Qualifierif your file has quotes or any other qualifier,
Now click on Next button, to see the preview to ensure that Agenty is reading the file correctly with settings you have applied,
If you see the data is populated correctly in table preview, click on the Next button,
See the final confirmation, and click on Final Upload button,
The list has been created and uploaded successfully. You can go back to the Lists menu to see the list and its content.
Select List in Agent
- Go to your agent page
- Click on the Input tab
- Now choose the "Select a URL List" option in Input Type
- Select your list in dropdown
- Finally, select the field which contains the URL in your list
- And Save the input configuration
- Now, we can start the web scraping job immediately, or schedule it to run later for selected list.
Update a List
- Go to the Lists page
- Click on the Action tab for the list you want to update
- Then select
Upload filefrom dropdown to add more URLs as we did in Create a List step.