c# - 使用GET从多个页面获取JSON数据

我需要从URL获取数据。


下面的代码给出错误“读取文件末尾”。该URL是正确的，可以粘贴到浏览器中并查看结果。几次我收到“超过时间限制”错误，不确定是否可以对此做任何事情。
问题是数据可以在几页上，我是否必须从page1中选择total_pages并进行循环？有更好的解决方案吗？


这是代码：

string url="https://jsonmock.hackerrank.com/api/movies/search/?Title=spiderman&page=1";
string res=MakeRequest(url);

MakeRequest：

static public string MakeRequest(string url)
{

 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = "application/json; charset=utf-8";
    request.PreAuthenticate = true;

 HttpWebResponse response = request.GetResponse() as HttpWebResponse;
    using (Stream responseStream = response.GetResponseStream())
    {
        StreamReader reader = new StreamReader(responseStream, Encoding.UTF8);
        return (reader.ReadToEnd());

    }

}

这是数据（第2页）的样子

{"page":"2","per_page":10,"total":13,"total_pages":2,"data":[{"Poster":"N/A","Title":"They Call Me Spiderman","Type":"movie","Year":2016,"imdbID":"tt5861236"},{"Poster":"N/A","Title":"The Death of Spiderman","Type":"movie","Year":2015,"imdbID":"tt5921428"},{"Poster":"https://images-na.ssl-images-amazon.com/images/M/MV5BZDlmMGQwYmItNTNmOS00OTNkLTkxNTYtNDM3ZWVlMWUyZDIzXkEyXkFqcGdeQXVyMTA5Mzk5Mw@@._V1_SX300.jpg","Title":"Spiderman in Cannes","Type":"movie","Year":2016,"imdbID":"tt5978586"}]}

最佳答案

使用HttpClient和Newtonsoft.Json的完整示例。

首先，让我们为数据传输对象定义类，以匹配API中的json对应对象：

public class PageResponse
{
    public string Page { get; set;}

    [JsonProperty("per_page")]
    public int PerPage { get; set;}

    public int Total { get; set; }

    [JsonProperty("total_pages")]
    public int TotalPages { get; set; }

    public IEnumerable<Movie> Data { get; set; }
}

public class Movie
{
    public string Poster { get; set; }
    public string Title { get; set; }
    public string Type { get; set; }
    public int Year { get; set; }

    [JsonProperty("imdbID")]
    public string ImdbId { get; set; }
}

然后，让我们创建API调用方法，该方法使用do-while周期并执行请求，直到我们检索所有页面或失败（由于某种原因）为止：

public static async Task<IEnumerable<Movie>> GetMoviesAsync()
{
    var movies = new List<Movie>();
    var url = "http://jsonmock.hackerrank.com/api/movies/search/?Title=spiderman";
    int currentPage = 1;
    int totalPages = 0;
    var nextUrl = $"{url}&page={currentPage}";

    using (var httpClient = new HttpClient())
    {
        do
        {
            HttpResponseMessage response = await httpClient.GetAsync(nextUrl);

            if (response.IsSuccessStatusCode)
            {
                string json = await response.Content.ReadAsStringAsync();
                var pageResponse = JsonConvert.DeserializeObject<PageResponse>(json);

                if (pageResponse != null && pageResponse.Data.Any())
                {
                    movies.AddRange(pageResponse.Data);
                    totalPages = pageResponse.TotalPages;

                    currentPage++;
                    nextUrl = $"{url}&page={currentPage}";
                }
                else
                {
                    break; // or throw exception
                }
            }
            else
            {
                break; // or throw exception
            }
        } while (currentPage < totalPages);
    }

    return movies;
}

最后，从我们的Main方法调用GetMoviesAsync：

static void Main(string[] args)
{
    IEnumerable<Movie> movies = GetMoviesAsync().GetAwaiter().GetResult();
    Console.WriteLine($"Retrieved {movies.Count()} movies.");
}

...在进行此编辑时，总共检索了13部电影：

Retrieved 13 movies.

另外请注意，最佳做法是访问修饰符，例如public总是写在static关键字之前。即

public static ...

不：

static public ...

编辑：更新，并在下面的评论中建议更多详细信息

EDIT2：通过省略ContinueWith调用来改进GetMoviesAsync（）