如何在 Web API 中维护请求的状态或队列

问题描述

我遇到过这样的情况，我必须在 Web API 方法中接收请求，将这些请求排队，然后将批量发送到数据库(Solr 实例).

I have situation, where I have to receive requests in a Web API method, queue those request and then send the bulk to a database (Solr instance).

我不太确定如何维护来自多个来源的一批请求.现在我将每个请求数据以 json 格式写入磁盘文件，稍后我将有一个 Windows 服务，通过文件夹读取所有文件，更新数据库并删除这些文件.

I am not really sure how do I maintain a batch of requests from multiple sources. For now I am writing each request data in json format to a file on disk, Later I will have a windows service, go through the folder read all files , update the database and delete those files.

这是我在 Web API 中所做的

Here is what I am doing in my Web API

public void Post(LogEntry value)
{
    value.EventID = Guid.NewGuid();
    value.ServerTime = DateTime.UtcNow;
    string json = JsonConvert.SerializeObject(value);
    using(StreamWriter sw = new StreamWriter(value.EventID.ToString()))
    {
        sw.Write(json);
    }
}

(这里EventID是GUID)

(Here EventID is GUID)

这个过程看起来不太对，肯定有办法维护一个请求队列，但是我不太确定如何在多个请求期间维护一个队列.

This process doesn't look right, there must be a way to maintain a queue of request, but I am not really sure how to maintain a queue during multiple requests.

我这样做的原因是，在 solr 实例中批量插入比通过 SolrNet 插入单个记录更快.我期望在 Web API 上每秒至少收到 100 个请求.我想创建一批 1000 个请求并每 10 秒更新一次 solr 实例.请不要认为我需要代码，只需要知道我应该采用什么策略来维护请求/状态队列.

The reason I am doing that is, insertion in batches in solr instance is faster than inserting a single record through SolrNet. I am expecting to get at least 100 requests each second on the Web API. I want to create a batch of 1000 request and update the solr instance every 10 seconds. Please don't think that I need code, just need to know what strategy should I adopt to maintain a queue of request / state.

推荐答案

如果您使用 .NET 4.0 或更高版本，您可以使用并发队列:

You could use a concurrent queue, if you're using .NET 4.0 or higher:

并发队列 (MSDN)

这是使用队列的线程安全方式，然后可以在所需的时间访问.

This is a thread-safe way of using a queue, which then could be accessed at a desired time.

示例:

这将是队列的包装器:

public static class RequestQueue
{
    private static ConcurrentQueue<int> _queue;

    public static ConcurrentQueue<int> Queue
    {
        get
        {
            if (_queue == null)
            {
                _queue = new ConcurrentQueue<int>();
            }

            return _queue;
        }
    }

}

然后你可以像这样设置你的 web api(为了简洁起见，这个例子存储了整数):

Then you could set up your web api like this (this example stores integers for the sake of brevity):

public class ValuesController : ApiController
{
    public string Get()
    {
        var sb = new StringBuilder();
        foreach (var item in RequestQueue.Queue)
        {
            sb.Append(item.ToString());
        }

        return sb.ToString();
    }

    public void Post(int id)
    {
        RequestQueue.Queue.Enqueue(id);
    }
}

如果你使用这个例子，你会看到队列保存了多个请求的值.但是，由于它存在于内存中，如果应用程序池被回收(例如)，那些排队的项目将消失.

If u use this example you'll see that the queue holds the values across multiple requests. But, since it lives in memory, those queued items will be gone if the app pool is recycled (for instance).

现在您可以检查队列何时包含 10 个项目，然后将它们保存到数据库，同时创建另一个队列来存储传入的值.

Now you could build in a check for when the queue holds 10 items and then save those to the DB, while creating another queue to store incoming values.

像这样:

public static class RequestQueue
{
    private static ConcurrentQueue<int> _queue;

    public static ConcurrentQueue<int> Queue
    {
        get
        {
            if (_queue == null)
            {
                _queue = new ConcurrentQueue<int>();
            }

            if (_queue.Count >= 10)
            {
                SaveToDB(_queue);
                _queue = new ConcurrentQueue<int>();
            }

            return _queue;
        }
    }

    public static void SaveToDB(ConcurrentQueue<int> queue)
    {
        foreach (var item in queue)
        {
            SaveItemToDB(item);
        }
    }
}

您需要稍微清理一下，但此设置应该可以工作.此外，您可能需要一些锁定机制来将队列转储到数据库并创建新实例.我会编写一个带有多个线程的控制台应用程序来访问这个队列来测试它.

You need to clean this up a bit, but this setup should work. Also, you might need some locking mechanism around the dumping of the queue to the DB and creating a new instance. I would write a Console app with multiple threads that access this Queue to test it.

这篇关于如何在 Web API 中维护请求的状态或队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！