* 겪고 있는 문제 상황을 최대한 자세하게 작성해주세요.
* 문제 해결을 위해 어떤 시도를 해보았는지 구체적으로 함께 알려주세요.
- 도서크롤링과 뉴스크롤링 수업을 보면 class 정보를 가지고 올떄 css_selector를 쓸떄도 있고, class_name을 쓸떄도 있는것 같습니다. 각각 기능의 차이점은 뭔가요?
- 도서 크롤링 페이지 실습을 하다보니 class 정의 되어있는게 바뀐거 같은데, 다른분 문의에 달아놓으신대로 수정해도 실행이 되지 않고 다음 처럼 페이지가 뜹니다. 무엇을 잘못한걸까요?
작성한 코드 및 에러 메세지
작성코드이고
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=chrome_options)
driver.get("https://product.kyobobook.co.kr/bestseller/online?period=001&page=1&per=50")
books_elements = driver.find_elements(By.CSS_SELECTOR, 'ol.grid > li')
for index, book in enumerate(books_elements):
rank = index + 1
title = book.find_element(By.CSS_SELECTOR, "a.prod_link.font-weight-medium").text
author_info = book.find_element(By.CSS_SELECTOR, "div.line-clamp-2.flex.overflow-hidden.whitespace-normal").text
author = author_info.split('·')[0].strip()
price = book.find_element(By.CSS_SELECTOR, "div.mt-3 > span:nth-child(2)").text
print(f"{rank}등 {title} by {author} 가격 : {price}")
driver.quit()
에러메세지
.venv) Seulgiui-MacBookAir:crawling seulgikim$ /Users/seulgikim/Desktop/automation/crawling/.venv/bin/python /Users/seulgikim/Desktop/automation/crawling/a.py
Traceback (most recent call last):
File "/Users/seulgikim/Desktop/automation/crawling/a.py", line 19, in <module>
price = book.find_element(By.CSS_SELECTOR, "div.mt-3 > span:nth-child(2)").text
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 416, in find_element
return self._execute(Command.FIND_CHILD_ELEMENT, {"using": by, "value": value})["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 394, in _execute
return self._parent.execute(command, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
self.error_handler.check_response(response)
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.mt-3 > span:nth-child(2)"}
(Session info: chrome=129.0.6668.101); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
0 chromedriver 0x000000010e76dd08 chromedriver + 4996360
1 chromedriver 0x000000010e7655ca chromedriver + 4961738
2 chromedriver 0x000000010e308d10 chromedriver + 388368
3 chromedriver 0x000000010e35530f chromedriver + 701199
4 chromedriver 0x000000010e3553f1 chromedriver + 701425
5 chromedriver 0x000000010e349d46 chromedriver + 654662
6 chromedriver 0x000000010e3799dd chromedriver + 850397
7 chromedriver 0x000000010e349c48 chromedriver + 654408
8 chromedriver 0x000000010e379b6e chromedriver + 850798
9 chromedriver 0x000000010e397a00 chromedriver + 973312
10 chromedriver 0x000000010e379753 chromedriver + 849747
11 chromedriver 0x000000010e348635 chromedriver + 648757
12 chromedriver 0x000000010e348e5e chromedriver + 650846
13 chromedriver 0x000000010e733ff0 chromedriver + 4759536
14 chromedriver 0x000000010e738f08 chromedriver + 4779784
15 chromedriver 0x000000010e7395d5 chromedriver + 4781525
16 chromedriver 0x000000010e716a99 chromedriver + 4639385
17 chromedriver 0x000000010e7398c9 chromedriver + 4782281
18 chromedriver 0x000000010e708034 chromedriver + 4579380
19 chromedriver 0x000000010e7559f8 chromedriver + 4897272
20 chromedriver 0x000000010e755bf3 chromedriver + 4897779
21 chromedriver 0x000000010e7651ce chromedriver + 4960718
22 libsystem_pthread.dylib 0x00007ff80e1e84e1 _pthread_start + 125
23 libsystem_pthread.dylib 0x00007ff80e1e3f6b thread_start + 15
(.venv) Seulgiui-MacBookAir:crawling seulgikim$ /Users/seulgikim/Desktop/automation/crawling/.venv/bin/python /Users/seulgikim/Desktop/automation/crawling/a.py
Traceback (most recent call last):
File "/Users/seulgikim/Desktop/automation/crawling/a.py", line 19, in <module>
price = book.find_element(By.CSS_SELECTOR, "div.mt-3 > span:nth-child(2)").text
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 416, in find_element
return self._execute(Command.FIND_CHILD_ELEMENT, {"using": by, "value": value})["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 394, in _execute
return self._parent.execute(command, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
self.error_handler.check_response(response)
File "/Users/seulgikim/Desktop/automation/crawling/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.mt-3 > span:nth-child(2)"}
(Session info: chrome=129.0.6668.101); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
0 chromedriver 0x000000010c92cd08 chromedriver + 4996360
1 chromedriver 0x000000010c9245ca chromedriver + 4961738
2 chromedriver 0x000000010c4c7d10 chromedriver + 388368
3 chromedriver 0x000000010c51430f chromedriver + 701199
4 chromedriver 0x000000010c5143f1 chromedriver + 701425
5 chromedriver 0x000000010c508d46 chromedriver + 654662
6 chromedriver 0x000000010c5389dd chromedriver + 850397
7 chromedriver 0x000000010c508c48 chromedriver + 654408
8 chromedriver 0x000000010c538b6e chromedriver + 850798
9 chromedriver 0x000000010c556a00 chromedriver + 973312
10 chromedriver 0x000000010c538753 chromedriver + 849747
11 chromedriver 0x000000010c507635 chromedriver + 648757
12 chromedriver 0x000000010c507e5e chromedriver + 650846
13 chromedriver 0x000000010c8f2ff0 chromedriver + 4759536
14 chromedriver 0x000000010c8f7f08 chromedriver + 4779784
15 chromedriver 0x000000010c8f85d5 chromedriver + 4781525
16 chromedriver 0x000000010c8d5a99 chromedriver + 4639385
17 chromedriver 0x000000010c8f88c9 chromedriver + 4782281
18 chromedriver 0x000000010c8c7034 chromedriver + 4579380
19 chromedriver 0x000000010c9149f8 chromedriver + 4897272
20 chromedriver 0x000000010c914bf3 chromedriver + 4897779
21 chromedriver 0x000000010c9241ce chromedriver + 4960718
22 libsystem_pthread.dylib 0x00007ff80e1e84e1 _pthread_start + 125
23 libsystem_pthread.dylib 0x00007ff80e1e3f6b thread_start + 15
(.venv) Seulgiui-MacBookAir:crawling seulgikim$
오류 발생 시, 작성한 코드 전체와 에러 메시지를 첨부해 주세요.
Tip 1) </> 아이콘을 눌러 코드박스를 만들어 보세요.
Tip 2) Ctrl+A(맥의 경우 Command+A) 단축키로 코드를 한 번에 선택할 수 있어요!
