python - 如何遍历下拉列表Scrapy

我正在抓取以下网站https://www.trollandtoad.com/magic-the-gathering/magic-2020-m20-/14878?Keywords=&min-price=&max-price=&items-pp=60&item-condition=&selected-cat=14878&sort-order=&page-no=1&view=list&Rarity=&Ruleset=&minMana=&maxMana=&minPower=&maxPower=&minToughness=&maxToughness=，我需要遍历下拉列表中的数量，直到数量到尽为止，以便确定剩余库存。我在其中放置一个计数器，以确定它在循环中运行了多少次以确定剩余的库存量，但它仅在循环中运行了一次。

# Function to parse needed data
    def parse(self, response):

        # For loop to run through html code until all needed data is scraped
        for data in response.css('div.card > div.row'):
            # import items from items.py
            item = DataItem()
            # Scrape Category name
            item["Category"] = data.css("div.col-12.prod-cat a::text").get()
            # Scrape card name
            item["Card_Name"]  = data.css("a.card-text::text").get()
            item["Stock"] = data.css("div.font-weight-bold.font-smaller.text-muted::text").get()
            if item["Stock"] == None:
                item["Stock"] = "In Stock"
            # For loop to run through all the buying information needed, skips first row
            for buying_option in data.css('div.buying-options-table div.row')[1:]:
                # Scrape seller, condition, and price
                item["Seller"] = buying_option.css('div.row.align-center.py-2.m-auto > div.col-3.text-center.p-1 > img::attr(title)').get()
                if item["Seller"] == "PRE ORDER":
                    item["Seller"] = "TrollAndToad Com"
                item["Condition"] = buying_option.css("div.col-3.text-center.p-1::text").get()
                num = 0
                for select in buying_option.css('select.w-100'): # Right here is where I am trying to determine the stock by looping through drop down lsit
                    num = num + 1
                item["Price"] = buying_option.css("div.col-2.text-center.p-1::text").get()
                # Return data
                yield item

最佳答案

XPath有一个非常简单的方法：

stock_quantity = row.xpath('//select[@name="qtyToBuy"]/option[last()]/@value').get()

关于python - 如何遍历下拉列表Scrapy，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/56894086/