I am scraping a Shopify product website for a brand. That brand has multiple product color variants and when I scrape the product details page, only the default color details is scraped.
How can I scrape all the color variants for each product in next row? Also the other product details should be repeated in next rows in CSV.
Yes, you can scrape the color variants and then repeat the other values in next rows folloing these steps -
For example, I will use this example HTML page to showcase which has 3 colors variants as an option to select while making a purchase - https://s.bootsnipp.com/iframe/7n1O8
Step 1 : Find the selector for colors, and add a field to scrape all colors first. In this case the selector is
.swatches .swatch:nth-of-type(2) input and then I used the attribute option to extract the value attribute from this input html element.
Step 2 : Add other fields and save your scraping agent and it will looks like this, by default -
Because 2 fields “product name” and “price” has 1 match, while the colors has 3 matches. So it’s taking 3 rows by default… .You can always use the JoinResult option to join more then one result together into one cell with comma separated(or some other delimiter) as in docs here - https://www.agenty.com/docs/scraping-agent/join-result
But since you want to have it in same format and name, price should be repeated to make your file importable in Shopify to create colors variants.
So, we are going to use the AutoFillBlankCells post-processing function on fields which should be repeated.
Step 3 : Add the auto fill blank cells function on fields which should be repeated.
Run it again after the changes in configuration and you’ll get the format you are looking for -