[問題] HttpWebRequest下載股票資料失敗

作者: a2975313 (GN)   2019-10-06 20:41:18
本人不才想下載股市資料,用瀏覽器都可開啟,但只要用HttpWebRequest就會有
"基礎連接已關閉: 傳送時發生未預期的錯誤。"的問題。
原本想說是request header沒填造成的,但按照chrome dev tool所觀察到的
header值填也是一樣...
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("
https://www.twse.com.tw/exchangeReport/MI_INDEX?response=html&date=20191003
&type=ALL");
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "text/html, application/xhtml+xml,
application/xml; q=0.9, */*; q=0.8";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
Edge/18.18362";
request.Method = "GET";
request.Referer = "no-referrer-when-downgrade";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(new Uri("https://www.twse.com.tw/"),
new Cookie("_ga", "GA1.3.1661386241.1570275885"));
request.CookieContainer.Add(new Uri("https://www.twse.com.tw/"),
new Cookie("_gid", "GA1.3.1577552003.1570275885"));
request.Headers.Add("Accept-Encoding", "gzip, deflate, br");
request.Headers.Add("Accept-Language", "zh-TW");
request.Headers.Add("Cache-Control", "max-age=0");
request.Headers.Add("Sec-Fetch-Mode", "navigate");
request.Headers.Add("Sec-Fetch-Site", "none");
request.Headers.Add("Sec-Fetch-User", "?1");
request.Headers.Add("Upgrade-Insecure-Requests", "1");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
作者: Shrugg (好柿花生)   2019-10-06 20:48:00
網站有防止攻擊,不能連續取資料,要設定每捉一筆停頓幾秒,如果只是要大盤資料,去yahoo財經網或透過api去下載
作者: SeanChiu (阿智)   2019-10-20 01:10:00
應該是TLS版本問題,請參考https://blog.darkthread.net/blog/webclient-and-tls12/

Links booklink

Contact Us: admin [ a t ] ucptt.com