Python简单爬虫
爬取链家二手房源信息
在三门等地区,都构建了全面的区域性战略布局,加强发展的系统性、市场前瞻性、产品创新能力,以专注、极致的服务理念,为客户提供网站制作、成都网站设计 网站设计制作按需设计网站,公司网站建设,企业网站建设,品牌网站建设,营销型网站,成都外贸网站建设,三门网站建设费用合理。
import requests import re from bs4 import BeautifulSoup import csv url = ['https://cq.lianjia.com/ershoufang/'] for i in range(2,101): url.append('https://cq.lianjia.com/ershoufang/pg%s/'%(str(i))) # 模拟谷歌浏览器 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36'} for u in url: r = requests.get(u,headers=headers) soup = BeautifulSoup(r.text,'lxml').find_all('li', class_='clear LOGCLICKDATA') for i in soup: ns = i.select('div[class="positionInfo"]')[0].get_text() region = ns.split('-')[1].replace(' ','').encode('gbk') rem = ns.split('-')[0].replace(' ','').encode('gbk') ns = i.select('div[class="houseInfo"]')[0].get_text() xiaoqu_name = ns.split('|')[0].replace(' ','').encode('gbk') huxing = ns.split('|')[1].replace(' ','').encode('gbk') pingfang = ns.split('|')[2].replace(' ','').encode('gbk') chaoxiang = ns.split('|')[3].replace(' ','').encode('gbk') zhuangxiu = ns.split('|')[4].replace(' ','').encode('gbk') danjia = re.findall("\d+",i.select('div[class="unitPrice"]')[0].string)[0] zongjia = i.select('div[class="totalPrice"]')[0].get_text().encode('gbk') out=open("/data/data.csv",'a') csv_write=csv.writer(out) data = [region,xiaoqu_name,rem,huxing,pingfang,chaoxiang,zhuangxiu,danjia,zongjia] csv_write.writerow(data) out.close()
数据结果
新闻名称:Python简单爬虫
网站网址:http://scyanting.com/article/gpdipd.html