1. 概述
携程网是中国领先的在线旅行服务公司，提供酒店预订、机票预订、旅游度假、商旅管理等服务。携程网上有大量的旅游景点和酒店信息，这些信息对于旅行者和旅游业者都有很大的价值。通过爬虫技术，我们可以从携程网上获取这些信息，并进行数据清洗、数据分析、数据可视化等操作，从而得到有用的洞察和建议。
2. 安装requests 库
在开始之前，请确保你已经安装了以下 Python 库：
requests：用于发送 HTTP 请求并获取网页内容。
你可以使用 pip 来安装这些库：
pip install requests 3. 爬取携程旅游网站数据
首先，我们需要确定要爬取的页面。 假设我们想要获取携程旅游网站上某个目的地的旅游信息。如下例如北京。
当前接口 链接 和 post 参数
url = 'https://m.ctrip.com/restapi/soa2/18109/json/getAttractionList?_fxpcqlniredt=09031015313388236487&x-traceID=09031015313388236487-1712974794650-8267936' data = {"index":1,"count":10,"sortType":1,"isShowAggregation":true,"districtId":1,"scene":"DISTRICT","pageId":"214062","traceId":"14f9745c-92ad-f5c5-07bb-171293c80647","extension":[{"name":"osVersion","value":"10"},{"name":"deviceType","value":"windows"}],"filter":{"filterItems":[]},"crnVersion":"2020-09-01 22:00:45","isInitialState":true,"head":{"cid":"09031015313388236487","ctok":"","cver":"1.0","lang":"01","sid":"8888","syscode":"09","auth":"","xsid":"","extension":[]}}4. 开始正式代码
headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 Edg/123.0.0.0', 'Cookie': 你的cookies } html = requests.post(url, headers=headers, json=data).json() attractionList = html['attractionList'] for attraction in attractionList: data = attraction['card'] commentCount = data['commentCount'] commentScore = data['commentScore'] coordinate = [data['coordinate']['latitude'], data['coordinate']['longitude']] coverImageUrl = data.get('coverImageUrl','') # 距离 distanceStr = data.get('distanceStr','') # 地点 displayField = data.get('displayField', None) heatScore = data.get('heatScore','') # 景点名 poiName = data['poiName'] isFree = data['isFree'] if isFree: price = 0 # 原价 marketPrice = 0 else: price = data.get('price',0) # 原价 marketPrice = data.get('marketPrice',0) # 类别信息 sightCategoryInfo = data.get('sightCategoryInfo','') # 标签 tagNameList = data.get('tagNameList','') # 5a sightLevelStr = data.get('sightLevelStr', None)5. 保存到csv
f = open('csv/全国各景点全.csv', 'w', encoding="utf-8", newline='') csvwrite = csv.writer(f)csvwrite.writerow(['城市', '景点名', '地点', '距离', '坐标', '评论数','评论分','热评分','封面','是否免费','价格','原价','类别信息','标签','是否5A']) csvwrite.writerow([city,poiName,displayField,distanceStr,coordinate,commentCount,commentScore,heatScore,coverImageUrl,isFree,price,marketPrice,sightCategoryInfo,tagNameList,sightLevelStr])6 .通过获取全国 city id，可请求全国景点数据并保存
全国景点数据csv 地址 https://download.csdn.net/download/britlee/89115745