Throttling features allows to add some (seconds) delay in your scraping agent. Because, the website scraping agent is blazing fast and can make multiple concurrent requests to make the most of the capability under your pricing plan and the power of cloud machines. However, with great power comes with great responsibility and the first-rule of website scraping is not to harm the target website. So, we can use the Throttling feature to add few seconds delay in sequential requests.

There are 2 types of Delay options : 

1. Fixed Delay
2. Random Delay

Fixed Delay

The fix number of seconds to wait in each sequential requests

Before : If you see the screenshot below, the throttling feature is not used in this agent, so there are no wait and the scraping agent is running continuously for all the inputs one-after-one.

Steps to configure "Throttling" with a agent

Steps:

1. Edit the agent by clicking on the "Edit" tab
2. Go to "Throttling" section and "Enable Request Throttling" switch
3. Select the "Delay Type" as "Fixed delay" from drop-down
4. Now enter the "seconds to delay" value
5. Then "Save" the scraping agent configuration.
   
   
    
6. Finally re-run you agent to apply the changes.
    

After:

We can see the agent logs to ensure the agent is waiting for given interval time before making each next request, as in screenshot below:

Random Delay

Random Delay option is used to add random number of seconds delay on each request. Agenty will automatically generate a random number between 0 and the value you enter in "Seconds of delay" option, and then use that number to wait between requests.

Follow the same steps as in "Fixed delay", but select the Delay type as "Random delay" as in screenshot below : 


After:

Now, if you see the agent logs throttling event, it says "waiting 7 seconds" on first URL and then "waiting 6 seconds" for 2nd URL. So, Agenty generated the random number between 0 and 10 and applied in throttling configuration to delay in consecutive requests.