arrow_back Why doesn't the find from BeautifulSoup find the tag?

1 vote
I can't figure out why it can't find the review tags. everything works for the other tags.

import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent

ua = UserAgent()

response = requests.get(
headers={'user-agent': f'{ua.random}'}).text
soup = BeautifulSoup(response, 'lxml')
feedback = soup.find('span', jsname='bN97Pc').text


Most likely this data is loaded by js and not given immediately
The names of tags and classes look like they are generated by some algorithm. Try not to open the URL in a browser, but save the downloaded content to a file and look there.

1 Answer

Best answer
0 votes
It's as simple as quantum physics. We use our own script to save the response of the quest to disk. Then you open the file in Notepad and see if the tag is there with the right class. In 99.9% of cases it is not. In this case we google mana on selenium. Or you can do without selenium. There are a lot of ways, the main thing is to have enough imagination.

Here's one option (it may not work for you, or it may work and break tomorrow - I don't know. )
import requests
from bs4 import BeautifulSoup
import json

response = requests.get(
headers={'user-agent':'Hacked by HottabXP!'})

soup = BeautifulSoup(response.text,"lxml")
dirty_comments = soup.find_all('script')[34].string # Методом тыка определяем, что json с коментами хранится
# в 35 теге <script> и преобразовываем данные внутри тега в строку
valid_json = dirty_comments[dirty_comments.find(' data:')+6:dirty_comments.find(', sideChannel')] # Вырезаем всё, что
# находится между data: (плюс 6 символов) и sideChannel

comments_json = json.loads(valid_json) # Тут уже работаем с обычным json

for comment in comments_json[0]:
user_comment = comment[4] # Комментарий пользователя
author_answer = comment[7][1] # Ответ на комментарий пользователя (может быть None)
author_answer = '' # Если None, тогда в author_answer помещаем пустую строку

ResultAfter updating to Android 12 (Samsung One UI), mail stopped opening emails in the app. Only the list is visible, it is impossible to read the emails.
After updating, it became terrible, the password is entered for 150 times, constantly loses connection with the Internet, because of this is not possible to send attachments, if nothing changes will have to demolish.
The app just doesn't come in with the right threshold. I copy it into the web and everything comes in. The old one was logging in, I fucking updated it(((
In principle, the application is good, but after an update ad unit began to be displayed in place of the first letter. This is very inconvenient - it significantly reduces the space for the list of emails, plus often out of habit you poke at the advertisement instead of the first email (perhaps this is the idea). Looking for an alternative
You, of course, think about it for a while. In the meantime, we'll delete your app and start using another one. You're wooden! I'm yelling at you! The end user says to you, "Guys, I don't watch or read your ads. It interferes with my email. And the answer is nonsense. Annoying mail. Now you can't even delete ads from your inbox. Slide works, but the cart doesn't. I'm switching to outlook. More convenient and no ads. Mail can keep that. Like the news column on my page.
Samsung Galaxy S21 - after updating to Android 12 the app stopped working. Blinks and does not open, or opens, but it is impossible to read the letters. Refers to the fact that the Android WebView app is not updated, however, everything is updated to the latest release.
After Mail Mail connected to the service VK, it got worse. I have been struggling with restoring access to my mail for a week now, since I forgot my password. But I am persistently sent to enter through VK.And all attempts are unsuccessful. Already there is no desire to simply work with the service. Dear developers, I ask you to return the opportunity to restore access, before connecting to VK!
After the update, the emails in the app don't open. What is interesting is that after checking the application on the phone, the phone writes that there is a problem with the application, a failure in the application and sends to the developer. DEVELOPER, HOW MANY PEOPLE DO YOU NEED TO WRITE TO YOU ABOUT THE PROBLEM, SO THAT YOU FIX THE PROBLEM?
A lot of people have already written that after updating to android 12 and one UI 4.0 on Samsung, the app stopped opening emails. Gives an error "Error in webview". You write that you have to write to support, but it is not possible to do this from the app. When you press the button "write to the developer" comes out the same error.
Hello, I have installed your app twice in the last month and uninstalled it both times. It is impossible to make a login, constantly writes connection. The developer should be thrown out into the cold, and leave it there.
The application does not work after updating to Android 12, more than a month the problem is not solved, claims and comments to the developers are not considered and not solved, it seems that the mail has lost its relevance. All one answer, write a letter to the developer, and to the point...... no answers and no solutions to problems.... . The collapse of Mail is coming ........ . Rating of the app is not relevant, most of the reviews are 1-2 stars..... and they have.........!!!
After the update, the "News" notifications started coming in. I need the mail and nothing more, I do not need a browser inside the mail. I put one star, as it is impossible to remove this imposed innovation!
Good afternoon. After the last update, the Samsung S21 stopped opening emails. When I try to open it, it says "Error in viewing webview". I can not even enter the section "Write to the developer", so I am writing here.
Doesn't open emails after an Android update. Writes error webview. Write to the developer, as answers to all MAIR. RU, it is impossible, because the same error pops up. Enough already reply in this way, I think that you have already figured out what the problem. Solve it already.
some emails are automatically sorted into the "mailing list" folder. it is impossible to delete this folder. it is inconvenient, i need to see all emails in one place, instead of browsing through folders. it is a disgusting application
Can't use the app, can't access my email, crashes on the first or second second of loading the contents of incoming emails. I tried uninstalling the app and reinstalling it.
Before that it was rated 5 Mail×VK is the beginning of the end More ads. Pulse notifications that are impossible to close. I won't write to support) the company is dying, and that's good)
I would also put minus one, I remember my password and I can not enter the mail which I actively use more than 11 years, because the phone number that requests mail for a long time does not exist anymore !!!! This number of the Ukrainian operator I do not have as 5 years! What a discussion! I keep my documents, important photos, correspondence, who will return it all? I do not remember that at least once the application asked for my phone number, what kind of stupid updates
Annoying ads that are impossible to remove. It was better before the last update. A few more times accidentally weave in these ads and I will have to give up this application, and maybe all the products of Mile, as it seems the owners are only interested in profits, and the users do not care.
Dear developers, you offer to write to support, but when you try to do this you get an error viewing webview. What kind of mockery is this? Solve the problem at last, how many letters to you, and no solution!
It's been a long time since the app has worked. It either gives a window suggesting to update Android Webview, or it gives a strobe effect (the screen starts flashing white rapidly). Android Webview has already been updated. The firmware on the S21 is also up to date. But the app is still not working. Sending a message to the developer from the app is also not possible because of these errors. Letters respectively also can not put. In general, added a bunch of features that do not work.
Dear Developers! Samsung S21Ultra 5G does not work. 12 android. Before the last update, everything was working fine, but now "webview error". Fix it quickly, there are a lot of competitors:). No way to write directly to the developer, the application does not work. Please do not write about communication with the developer, but think about how to help users of the application and fix bugs.
The app sends ads in the phone notifications, but the notification is not in the program itself, or in the mail. This has never happened before. Uninstall this app, and it's better to check your email through the built-in app.
Well. Congratulations, comrades. Two weeks of waiting for the problem to be solved, and the problem does not even think to be solved. I will not write anywhere, I've had enough of writing to the developer for the rest of my life. Bottom line: you can not solve anything, you did a very good job, so that I deleted your application and went to Yandex.mails. all I advise, no need to keep it and the extra memory clutter. Yandex is our everything)).
Wrote to support about the problem of non-opening of letters after the update, went 2 weeks, no reaction. What work should be evaluated if nothing is done, the mail still does not work?
Well, do not you understand that all users of Samsung s 21, and probably other devices on 12m andrewshe can not use the application. Logs and other errors can not be sent because the application in the process of starting a stupid flickers on the splash screen image
Waiting for an update. Thought the app would still work on Android 12 and One UI 4.0, but it didn't! It still crashes. I can't even log in. Just a blinking screen with the initial splash screen.
... This is a device of mine that works every once in a while, I have not looked at the cause. Most likely you need to check the status of the response code of the quest. On the site you can conveniently see the json structure: 61cd9c0aca8c8860493418.png