Need help with Python script on importing user-posted genres tag from dlsite.com by html scrapping.

sonicstream

New member
May 18, 2022
5
1
I have hundreds of doujin folders to manage so I wrote python script that output excel file where all doujin folders are hyperlinked for easier access and can be sort by author and name.

I'm a novice in Python programming so I compiled the script with help of ChatGPT, but then while the script is able to extract genres from official tag, the script cannot extract genres that is voted by user reviews.

My code is compiled so to collect text embedded in <a href="/maniax/fsr/=genre/\d+/"></a> and <li class="meny selected "></li>, combining them into one line of output string text.

While <a href="/maniax/fsr/=genre/\d+/"></a> worked as intended in extracting work genres, <li class="meny selected "></li> failed to extract review voted work genres.

What else do i need to modify?

for url in product_urls:
product_link = url['href']
if "product_id" in product_link: # product_idが含まれるリンクだけを対象とする
product_response = requests.get(product_link)
product_soup = BeautifulSoup(product_response.content, "html.parser")

# /maniax/fsr/=/genre/*番号*/ のパターンを持つリンクを取得(class="btn_default"の有無に関わらず)
genre_links = product_soup.find_all("a", href=re.compile(r'/maniax/fsr/=/genre/\d+/'))

# 取得したリンクから<a>タグのテキスト内容を抽出し、genres集合に追加
for genre_link in genre_links:
genres.add(genre_link.text)

# <ul class="meny_selected_list">の中の<li class="many_selected_item">を探す
selected_items = product_soup.select("ul.meny_selected_list > li.many_selected_item")

# 各<li>タグ内の<a>タグのテキストをgenres集合に追加
for item in selected_items:
if item.a: # <a>タグが存在する場合
genres.add(item.a.text)

# ジャンル名をカンマで区切って1つの文字列に連結
if genres:
genres_text = ", ".join(genres)
 

Attachments

  • 同人作品ファイルリスト整理_タグ追加.zip
    2.5 KB · Views: 70
Last edited:

Users who are viewing this thread

Latest profile posts

vn88 là trang web cá cược trực tuyến uy tín, sở hữu nền tảng giải trí phong phú và chất lượng.
lulu464646 wrote on Ryzen111's profile.
Hi can you reupload this?
[170128][貴様巻きソフト] 超内気で人見知りでエロゲ好きな女の子が異世界に召喚された話 増量版 [v18.02.10 (v1.01)]
Thank you!
taatat wrote on Nihonjaki90's profile.
please reupload
http://www.anime~sharing.com\threads\221223-ガールズソフトウェア-エロキャス!-〜パパ活ギャルの罠〜-voice-drama-crack.1257976/
http://www.anime-sharing.com/threads/221028-ガールズソフトウェア-エロキャス!〜実況!淫乱保険医〜-crack.1238147/
http://www.anime-sharing.com\threads/230224-ガールズソフトウェア-エロキャス!-〜学級委員の隠れた性癖〜.1282807/
CowSlayer69 wrote on MsFangtasia's profile.
Happy Birthday :cool: