+1 vote
by
Why do I have only the first nickname on all 100 pages, although I should have 15 on each page
import requests
from bs4 import BeautifulSoup
from lxml import html

ses = requests.Session()
data = {"name": "vaseaxxx", "password": "vaseaxxx"}

auth = ses.post("http://mrush.mobi/login", data=data)

#print(auth.text)

for i in range(1,100):
mrush = open('ParsNick.txt','a')
html_text = ses.get(f"http://mrush.mobi/best?pvp=0&page={i}").text
#print(html_text)
soup = BeautifulSoup(html_text, 'html.parser')
names = soup.find_all('table', {"class": "cntr wa mlra"})
for name in names:
nick = name.a.text
print(nick)
mrush.write(nick+'\n')
mrush.close()
5e7c73d8a911d106063561.png

2 Answers

0 votes
by
 
Best answer
1) You are looking for tables with the class 'cntr wa mlra', and you need tables.
2) Already in the table found, search (as an option) for all 'a' tags with the class 'lwhite'
table = soup.find('table', {"class": "cntr wa mlra"})
names = table.find_all('a',class_='lwhite')
for name in names:
nick = name.text
print(nick)
3) You've already pissed them off, they put a clever protection against such scripts. You need to make delays when sending requests. About protection - now you need to pass one more parameter (with empty value) with random name during authorization. But just before authorization you can't just make a get request and pull this parameter out, it's a bit trickier there.
Working code (cookies copied from the browser):
import requests
from bs4 import BeautifulSoup
from lxml import html
import time

cookies = {'PHPSESSID':'c3a1cde86c8c8c9f0e3877403ad4935e.1585230344.54638591'}

headers = {'user-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0'}

for i in range(1,6):
mrush = open('ParsNick.txt','a')
html_text = requests.get(f"http://mrush.mobi/best?pvp=0&page={i}",cookies=cookies,headers=headers).text

soup = BeautifulSoup(html_text, 'html.parser')
table = soup.find('table', {"class": "cntr wa mlra"})
names = table.find_all('a',class_='lwhite')
for name in names:
nick = name.text
print(nick)
mrush.write(nick+'\n')
mrush.close()
time.sleep(1)
0 votes
by
Because you have to paginate
by
pass on
by
and how to do it ?
...