[問題] 網頁爬蟲問題

作者: ntasop (kuli)   2017-11-11 12:28:29
請問大家用post方式抓網頁資料,同樣方式加request header和json資料,
python可以正常抓到網路資料,但C#沒辦法,請問大家有甚麼地方還需要
注意,非常謝謝。
C#版本:
HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Add("referer", "https://shopee.tw/jeanch911");
httpClient.DefaultRequestHeaders.Add("user-agent", "Chrome/61.0.3163.100 Safari/537.36");
httpClient.DefaultRequestHeaders.Add("x-csrftoken", "iGwDsBPnVwMlr68yxp23vxYB3N53gMRH");
httpClient.DefaultRequestHeaders.Add("cookie", "略過太長....");
httpClient.BaseAddress = new Uri("https://shopee.tw/");
string str = "{\"shop_ids\":[6550848]};
JObject jsonObject = JObject.Parse(str);
var content = new StringContent(jsonObject.ToString(), Encoding.UTF8, "application/json");
var respone = await httpClient.PostAsync("api/v1/shops", content);
System.Console.WriteLine(respone.Content.ReadAsStringAsync().Result);
python版本:
headers = {
'Cookie':'.......略過太長',
'Referer':'https://shopee.tw/trc.tw',
'User-Agent':'Chrome/61.0.3163.100 Safari/537.36',
'X-CSRFToken':'3EwwiZKHrh74ZjKB1oHycwOgS6COQNon'}
jd = json.loads('{"shop_ids":[24550283]}')
response = requests.post('https://shopee.tw/api/v1/shops/', json = jd, headers = headers)
print(response.text.encode('utf-8').decode('unicode_escape'))
作者: vi000246 (Vi)   2017-11-11 13:22:00
抓封包看python跟c#有什麼不同
作者: ntasop (kuli)   2017-11-11 16:52:00
請問vi000246大,因沒學過這部分知識,該如何抓封包?
作者: james732 (好人超)   2017-11-11 18:15:00
用wireshark之類的工具吧?
作者: name2name2 (yang~hi)   2017-11-12 10:49:00
用httpclient.PostAsJsonAsync 試試
作者: vi000246 (Vi)   2017-11-12 20:08:00
chrome的F12、Fiddler、wireshark
作者: groovy2016   2017-11-13 01:44:00
firebug postman

Links booklink

Contact Us: admin [ a t ] ucptt.com