Agents API

Agents API is used to manage agents under an account. Use this API to fetch all agents, fetch agent by id, clone an agent, delete an agent etc.

Get all agents

This API fetches all the active agents under an account.

Endpoint:

Method: GET
URL: https://api.agenty.com/v1/agents

Query params:

Key Value Description
apikey {{API_KEY}}

Responses:

Status: OK | Code: 200

{
    "offset": 0,
    "limit": 1000,
    "returned": 3,
    "total": 3,
    "result": [
        {
            "agent_id": "7229mv10op",
            "project_id": null,
            "name": "Books price scraping agent",
            "description": "This agent will extract the product list, prices, image and detail page hyperlink from books.toscrape.com website",
            "type": "scraping",
            "version": 3,
            "created_at": "2019-03-08T02:12:11",
            "updated_at": "2019-03-08T02:34:09",
            "cron_expression": "0 0 8 1/1 * ? *",
            "schedule_description": "At 08:00 AM, every day",
            "next_auto_run_at": "2019-03-08T08:00:00",
            "access_group_id": 1
        },
        {
            "agent_id": "dg4ely7e9r",
            "project_id": null,
            "name": "eCommerce product scraper",
            "description": "This agent will scrape the demo product and prices form the source url.",
            "type": "scraping",
            "version": 2,
            "created_at": "2019-03-08T02:21:40",
            "updated_at": "2019-03-08T02:23:43",
            "cron_expression": null,
            "schedule_description": null,
            "next_auto_run_at": null,
            "access_group_id": 1
        },
        {
            "agent_id": "rzpger0e2e",
            "project_id": null,
            "name": "OCR Agent",
            "description": "The Agent will extract the text from images using optical recognition",
            "type": "ocr",
            "version": 3,
            "created_at": "2019-03-08T02:24:58",
            "updated_at": "2019-03-08T02:29:56",
            "cron_expression": null,
            "schedule_description": null,
            "next_auto_run_at": null,
            "access_group_id": 1
        }
    ]
}

Get agent by agent id

This API will fetch the agent by given agent id.

Endpoint:

Method: GET
URL: https://api.agenty.com/v1/agents/scraping/{{AGENT_ID}}

Query params:

Key Value Description
apikey {{API_KEY}}

Responses:

Status: Ok | Code: 200

{
    "agent_id": "7229mv10op",
    "project_id": null,
    "name": "Books price scraping agent",
    "description": "This agent will extract the product list, prices, image and detail page hyperlink from books.toscrape.com website",
    "type": "scraping",
    "version": 3,
    "created_at": "2019-03-08T02:12:11",
    "updated_at": "2019-03-08T05:24:59",
    "cron_expression": "0 0 8 1/1 * ? *",
    "schedule_description": "At 08:00 AM, every day",
    "next_auto_run_at": "2019-03-08T08:00:00",
    "access_group_id": 0,
    "config": {
        "sourceurl": "http://books.toscrape.com/",
        "collections": [
            {
                "name": "Collection1",
                "fields": [
                    {
                        "name": "NAME",
                        "type": "CSS",
                        "selector": "h3 a",
                        "extract": "TEXT",
                        "attribute": null,
                        "from": null,
                        "visible": true,
                        "cleantrim": true,
                        "joinresult": false,
                        "postprocessing": [],
                        "formatter": []
                    },
                    {
                        "name": "PRICE",
                        "type": "CSS",
                        "selector": ".price_color",
                        "extract": "TEXT",
                        "attribute": "",
                        "from": null,
                        "visible": true,
                        "cleantrim": true,
                        "joinresult": false,
                        "postprocessing": [
                            {
                                "function": "Insert",
                                "parameters": [
                                    {
                                        "name": null,
                                        "value": "http://books.toscrape.com/"
                                    }
                                ]
                            }
                        ],
                        "formatter": []
                    },
                    {
                        "name": "IMAGE",
                        "type": "CSS",
                        "selector": ".thumbnail",
                        "extract": "ATTR",
                        "attribute": "src",
                        "from": null,
                        "visible": true,
                        "cleantrim": true,
                        "joinresult": false,
                        "postprocessing": [
                            {
                                "function": "Insert",
                                "parameters": [
                                    {
                                        "name": "Input",
                                        "value": "http://books.toscrape.com/"
                                    }
                                ]
                            }
                        ],
                        "formatter": []
                    },
                    {
                        "name": "DETAILS_PAGE_URL",
                        "type": "CSS",
                        "selector": ".product_pod h3 a",
                        "extract": "ATTR",
                        "attribute": "href",
                        "from": null,
                        "visible": true,
                        "cleantrim": true,
                        "joinresult": false,
                        "postprocessing": [
                            {
                                "function": "Insert",
                                "parameters": [
                                    {
                                        "name": "Input",
                                        "value": "http://books.toscrape.com/"
                                    }
                                ]
                            }
                        ],
                        "formatter": []
                    }
                ]
            }
        ],
        "engine": {
            "name": "default",
            "loadjavascript": false,
            "loadimages": false,
            "timeout": 30,
            "viewport": {
                "width": 1280,
                "height": 600
            }
        },
        "waitafterpageload": null,
        "login": {
            "enabled": false,
            "type": null,
            "data": []
        },
        "logout": null,
        "pagination": {
            "enabled": false,
            "type": null,
            "selector": null,
            "maxpages": 0
        },
        "header": {
            "method": "GET",
            "encoding": "utf-8",
            "data": [
                {
                    "key": "Accept",
                    "value": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
                },
                {
                    "key": "User-Agent",
                    "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
                },
                {
                    "key": "Accept-Language",
                    "value": "*"
                }
            ]
        },
        "autoredirect": {
            "enabled": true,
            "maxautoredirect": 3
        },
        "failretry": {
            "enabled": true,
            "maxtry": 3,
            "tryinterval": 2,
            "timeout": 0
        },
        "proxy": {
            "enabled": false,
            "type": null,
            "reference": null
        },
        "throttling": {
            "enabled": false,
            "type": null,
            "seconds": 0
        },
        "formsubmit": {
            "enabled": false,
            "data": []
        },
        "profiles": {
            "enabled": false,
            "data": []
        },
        "meta": null,
        "input": {
            "type": "MANUAL",
            "reference": null
        }
    }   
}

Clone agent by id

This API will clone the agent and also copy the input URLs if input type is manual or source URL.

Endpoint:

Method: GET
URL: https://api.agenty.com/v1/agents/{{AGENT_ID}}/clone

Query params:

Key Value Description
apikey {{API_KEY}}

Responses:

Status: OK | Code: 200

{
    "status_code": 200,
    "message": "A new scraping agent created successfully with id: vk97v1p30p",
    "type": "scraping",
    "agent_id": "vk97v1p30p"
}

Delete agent by id

This API will delete the agent permanently by given agent id.

Endpoint:

Method: DELETE
URL: https://api.agenty.com/v1/agents/{{AGENT_ID}}

Query params:

Key Value Description
apikey {{API_KEY}}

Responses:

Status: OK | Code: 200

{
    "status_code": 200,
    "message": "1 agent(s) deleted successfully"
}