统一流控服务开源:基于.Net Core的流控服务

 

先前有一篇博文,梳理了流控服务的场景、业界做法和常用算法

统一流控服务开源-1:场景&业界做法&算法篇

最近完成了流控服务的开发,并在生产系统进行了大半年的验证,稳定可靠。今天整理一下核心设计和实现思路,开源到Github上,分享给大家

Net Core的流控服务-LMLPHP

https://github.com/zhouguoqing/FlowControl

Net Core的流控服务-LMLPHP

 一、令牌桶算法实现

先回顾一下令牌桶算法示意图

Net Core的流控服务-LMLPHP

随着时间流逝,系统会按恒定1/QPS时间间隔(如果QPS=100,则间隔是10ms) 往桶里加入Token(想象和漏洞漏水相反,有个水龙头在不断的加水),

如果桶已经满了就不再加了. 新请求来临时, 会各自拿走一个Token,如果没有Token可拿了就阻塞或者拒绝服务.

令牌添加速度支持动态变化,实时控制处理的速率.

令牌桶有两个关键的属性:令牌桶容量(大小)和时间间隔,

有两个关键操作,从令牌桶中取Token;令牌桶定时的Reset重置。

  我们看TokenBucket类:

Net Core的流控服务-LMLPHP
using System;

namespace CZ.FlowControl.Service
{
using CZ.FlowControl.Spi;
/// <summary>
/// 令牌桶
/// </summary>
public abstract class TokenBucket : IThrottleStrategy
{
protected long bucketTokenCapacity;
private static readonly object syncRoot = new object();
protected readonly long ticksRefillInterval;
protected long nextRefillTime; //number of tokens in the bucket
protected long tokens; protected TokenBucket(long bucketTokenCapacity, long refillInterval, long refillIntervalInMilliSeconds)
{
if (bucketTokenCapacity <= 0)
throw new ArgumentOutOfRangeException("bucketTokenCapacity", "bucket token capacity can not be negative");
if (refillInterval < 0)
throw new ArgumentOutOfRangeException("refillInterval", "Refill interval cannot be negative");
if (refillIntervalInMilliSeconds <= 0)
throw new ArgumentOutOfRangeException("refillIntervalInMilliSeconds", "Refill interval in milliseconds cannot be negative"); this.bucketTokenCapacity = bucketTokenCapacity;
ticksRefillInterval = TimeSpan.FromMilliseconds(refillInterval * refillIntervalInMilliSeconds).Ticks;
} /// <summary>
/// 是否流控
/// </summary>
/// <param name="n"></param>
/// <returns></returns>
public bool ShouldThrottle(long n = 1)
{
TimeSpan waitTime;
return ShouldThrottle(n, out waitTime);
}
public bool ShouldThrottle(long n, out TimeSpan waitTime)
{
if (n <= 0) throw new ArgumentOutOfRangeException("n", "Should be positive integer"); lock (syncRoot)
{
UpdateTokens();
if (tokens < n)
{
var timeToIntervalEnd = nextRefillTime - SystemTime.UtcNow.Ticks;
if (timeToIntervalEnd < 0) return ShouldThrottle(n, out waitTime); waitTime = TimeSpan.FromTicks(timeToIntervalEnd);
return true;
}
tokens -= n; waitTime = TimeSpan.Zero;
return false;
}
} /// <summary>
/// 更新令牌
/// </summary>
protected abstract void UpdateTokens(); public bool ShouldThrottle(out TimeSpan waitTime)
{
return ShouldThrottle(1, out waitTime);
} public long CurrentTokenCount
{
get
{
lock (syncRoot)
{
UpdateTokens();
return tokens;
}
}
}
}
}
Net Core的流控服务-LMLPHP

这个抽象类中,将UpdateToken作为抽象方法暴露出来,给实现类更多的灵活去控制令牌桶重置操作。基于此实现了“固定令牌桶”FixedTokenBucket

Net Core的流控服务-LMLPHP
    /// <summary>
/// 固定令牌桶
/// </summary>
class FixedTokenBucket : TokenBucket
{
public FixedTokenBucket(long maxTokens, long refillInterval, long refillIntervalInMilliSeconds)
: base(maxTokens, refillInterval, refillIntervalInMilliSeconds)
{
} protected override void UpdateTokens()
{
var currentTime = SystemTime.UtcNow.Ticks; if (currentTime < nextRefillTime)
return; tokens = bucketTokenCapacity;
nextRefillTime = currentTime + ticksRefillInterval;
}
}
Net Core的流控服务-LMLPHP

固定令牌桶在每次取Token时,都要执行方法ShouldThrottle。这个方法中:

并发取Token是线程安全的,这个地方用了Lock控制,损失了一部分性能。同时每次获取可用Token的时候,都会实时Check一下是否需要到达Reset令牌桶的时间。

获取到可用令牌后,令牌桶中令牌的数量-1。如果没有足够的可用令牌,则返回等待到下次Reset令牌桶的时间。如下代码:

Net Core的流控服务-LMLPHP
        public bool ShouldThrottle(long n, out TimeSpan waitTime)
{
if (n <= 0) throw new ArgumentOutOfRangeException("n", "Should be positive integer"); lock (syncRoot)
{
UpdateTokens();
if (tokens < n)
{
var timeToIntervalEnd = nextRefillTime - SystemTime.UtcNow.Ticks;
if (timeToIntervalEnd < 0) return ShouldThrottle(n, out waitTime); waitTime = TimeSpan.FromTicks(timeToIntervalEnd);
return true;
}
tokens -= n; waitTime = TimeSpan.Zero;
return false;
}
}
Net Core的流控服务-LMLPHP

以上就是令牌桶算法的实现。我们继续看漏桶算法。

 二、漏桶算法实现

首先回顾一下漏桶算法的原理:

Net Core的流控服务-LMLPHP

水(请求)先进入到漏桶里,漏桶以一定的速度出水(接口有响应速率),

当水流入速度过大会直接溢出(访问频率超过接口响应速率), 然后就拒绝请求,

可以看出漏桶算法能强行限制数据的传输速率.

有两个变量:

  • 一个是桶的大小,支持流量突发增多时可以存多少的水(burst),
  • 另一个是水桶漏洞的大小(rate)。

漏桶抽象类:LeakTokenBucket,继承与令牌桶抽象父类 TokenBucket,说明了获取令牌(漏出令牌)在底层的方式是一致的,不一样的是重置令牌的方式(务必理解这一点)

Net Core的流控服务-LMLPHP
using System;

namespace CZ.FlowControl.Service
{
/// <summary>
/// 漏桶
/// </summary>
abstract class LeakyTokenBucket : TokenBucket
{
protected readonly long stepTokens;
protected long ticksStepInterval; protected LeakyTokenBucket(long maxTokens, long refillInterval, int refillIntervalInMilliSeconds,
long stepTokens, long stepInterval, int stepIntervalInMilliseconds)
: base(maxTokens, refillInterval, refillIntervalInMilliSeconds)
{
this.stepTokens = stepTokens;
if (stepInterval < 0) throw new ArgumentOutOfRangeException("stepInterval", "Step interval cannot be negative");
if (stepTokens < 0) throw new ArgumentOutOfRangeException("stepTokens", "Step tokens cannot be negative");
if (stepIntervalInMilliseconds <= 0) throw new ArgumentOutOfRangeException("stepIntervalInMilliseconds", "Step interval in milliseconds cannot be negative"); ticksStepInterval = TimeSpan.FromMilliseconds(stepInterval * stepIntervalInMilliseconds).Ticks;
}
}
}
Net Core的流控服务-LMLPHP

可以看出,漏桶是在令牌桶的基础上增加了二个重要的属性:这两个属性决定了重置令牌桶的方式

stepTokens:每间隔时间内漏的数量

ticksStepInterval:漏的间隔时间

举个例子:TPS 100,即每秒漏出100个Token,stepTokens =100, ticksStepInterval=1000ms

漏桶的具体实现有两种:空桶和满桶

StepDownTokenBucket 满桶:即一把将令牌桶填充满

 

StepUpLeakyTokenBucket 空桶:即每次只将stepTokens个数的令牌放到桶中

 

三、流控服务封装

第二章节,详细介绍了令牌桶和漏桶的具体实现。基于以上,要重点介绍接口:IThrottleStrategy:流控的具体方式

Net Core的流控服务-LMLPHP
using System;

namespace CZ.FlowControl.Spi
{
/// <summary>
/// 流量控制算法策略
/// </summary>
public interface IThrottleStrategy
{
/// <summary>
/// 是否流控
/// </summary>
/// <param name="n"></param>
/// <returns></returns>
bool ShouldThrottle(long n = 1); /// <summary>
/// 是否流控
/// </summary>
/// <param name="n"></param>
/// <param name="waitTime"></param>
/// <returns></returns>
bool ShouldThrottle(long n, out TimeSpan waitTime); /// <summary>
/// 是否流控
/// </summary>
/// <param name="waitTime"></param>
/// <returns></returns>
bool ShouldThrottle(out TimeSpan waitTime); /// <summary>
/// 当前令牌个数
/// </summary>
long CurrentTokenCount { get; }
}
}
Net Core的流控服务-LMLPHP

有了这个流控方式接口后,我们还需要一个流控策略定义类:FlowControlStrategy

即定义具体的流控策略:以下是这个类的详细属性和成员:  不仅定义了流控策略类型,还定义了流控的维度信息和流控阈值,这样流控就做成依赖注入的方式了!

Net Core的流控服务-LMLPHP
using System;
using System.Collections.Generic;
using System.Text; namespace CZ.FlowControl.Spi
{
/// <summary>
/// 流控策略
/// </summary>
public class FlowControlStrategy
{
/// <summary>
/// 标识
/// </summary>
public string ID { get; set; } /// <summary>
/// 名称
/// </summary>
public string Name { get; set; } /// <summary>
/// 流控策略类型
/// </summary>
public FlowControlStrategyType StrategyType { get; set; } /// <summary>
/// 流控阈值-Int
/// </summary>
public long IntThreshold { get; set; } /// <summary>
/// 流控阈值-Double
/// </summary>
public decimal DoubleThreshold { get; set; } /// <summary>
/// 时间区间跨度
/// </summary>
public FlowControlTimespan TimeSpan { get; set; } private Dictionary<string, string> flowControlConfigs; /// <summary>
/// 流控维度信息
/// </summary>
public Dictionary<string, string> FlowControlConfigs
{
get
{
if (flowControlConfigs == null)
flowControlConfigs = new Dictionary<string, string>(); return flowControlConfigs;
}
set
{
flowControlConfigs = value;
}
} /// <summary>
/// 描述
/// </summary>
public string Descriptions { get; set; } /// <summary>
/// 触发流控后是否直接拒绝请求
/// </summary>
public bool IsRefusedRequest { get; set; } /// <summary>
/// 创建时间
/// </summary>
public DateTime CreateTime { get; set; } /// <summary>
/// 创建人
/// </summary>
public string Creator { get; set; } /// <summary>
/// 最后修改时间
/// </summary>
public DateTime LastModifyTime { get; set; } /// <summary>
/// 最后修改人
/// </summary>
public string LastModifier { get; set; }
}
}
Net Core的流控服务-LMLPHP

同时,流控策略类型,我们抽象了一个枚举:FlowControlStrategyType

支持3种流控策略:TPS、Sum(指定时间段内请求的次数),Delay延迟

Net Core的流控服务-LMLPHP
using System;
using System.Collections.Generic;
using System.Text; namespace CZ.FlowControl.Spi
{
/// <summary>
/// 流控策略类型枚举
/// </summary>
public enum FlowControlStrategyType
{
/// <summary>
/// TPS控制策略
/// </summary>
TPS,
/// <summary>
/// 总数控制策略
/// </summary>
Sum, /// <summary>
/// 延迟控制策略
/// </summary>
Delay
}
}
Net Core的流控服务-LMLPHP

面向每种流控策略类型,提供了一个对应的流控器,比如说TPS的流控器

TPSFlowController,内部使用了固定令牌桶算法
Net Core的流控服务-LMLPHP
using System;

namespace CZ.FlowControl.Service
{
using CZ.FlowControl.Spi; /// <summary>
/// TPS流量控制器
/// </summary>
class TPSFlowController : IFlowController
{
public IThrottleStrategy InnerThrottleStrategy
{
get; private set;
} public FlowControlStrategy FlowControlStrategy { get; private set; } public bool ShouldThrottle(long n, out TimeSpan waitTime)
{
return InnerThrottleStrategy.ShouldThrottle(n, out waitTime);
} public TPSFlowController(FlowControlStrategy strategy)
{
FlowControlStrategy = strategy; InnerThrottleStrategy = new FixedTokenBucket(strategy.IntThreshold, 1, 1000);
}
}
}
Net Core的流控服务-LMLPHP

Sum(指定时间段内请求的次数)流控器:

Net Core的流控服务-LMLPHP
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text; namespace CZ.FlowControl.Service
{
using CZ.FlowControl.Spi; /// <summary>
/// 一段时间内合计值流量控制器
/// </summary>
class SumFlowController : IFlowController
{
public IThrottleStrategy InnerThrottleStrategy
{
get; private set;
} public FlowControlStrategy FlowControlStrategy { get; private set; } public bool ShouldThrottle(long n, out TimeSpan waitTime)
{
return InnerThrottleStrategy.ShouldThrottle(n, out waitTime);
} public SumFlowController(FlowControlStrategy strategy)
{
FlowControlStrategy = strategy; var refillInterval = GetTokenBucketRefillInterval(strategy); InnerThrottleStrategy = new FixedTokenBucket(strategy.IntThreshold, refillInterval, 1000);
} private long GetTokenBucketRefillInterval(FlowControlStrategy strategy)
{
long refillInterval = 0; switch (strategy.TimeSpan)
{
case FlowControlTimespan.Second:
refillInterval = 1;
break;
case FlowControlTimespan.Minute:
refillInterval = 60;
break;
case FlowControlTimespan.Hour:
refillInterval = 60 * 60;
break;
case FlowControlTimespan.Day:
refillInterval = 24 * 60 * 60;
break;
} return refillInterval;
}
}
}
Net Core的流控服务-LMLPHP

同时,通过一个创建者工厂,根据不同的流控策略,创建对应的流控器(做了一层缓存,性能更好):

Net Core的流控服务-LMLPHP
using System;
using System.Collections.Generic;
using System.Text; namespace CZ.FlowControl.Service
{
using CZ.FlowControl.Spi; /// <summary>
/// 流控策略工厂
/// </summary>
class FlowControllerFactory
{
private static Dictionary<string, IFlowController> fcControllers;
private static object syncObj = new object(); private static FlowControllerFactory instance; private FlowControllerFactory()
{
fcControllers = new Dictionary<string, IFlowController>();
} public static FlowControllerFactory GetInstance()
{
if (instance == null)
{
lock (syncObj)
{
if (instance == null)
{
instance = new FlowControllerFactory();
}
}
} return instance;
} public IFlowController GetOrCreateFlowController(FlowControlStrategy strategy)
{
if (strategy == null)
throw new ArgumentNullException("FlowControllerFactory.GetOrCreateFlowController.strategy"); if (!fcControllers.ContainsKey(strategy.ID))
{
lock (syncObj)
{
if (!fcControllers.ContainsKey(strategy.ID))
{
var fcController = CreateFlowController(strategy);
if (fcController != null)
fcControllers.Add(strategy.ID, fcController);
}
}
} if (fcControllers.ContainsKey(strategy.ID))
{
var controller = fcControllers[strategy.ID];
return controller;
} return null;
} private IFlowController CreateFlowController(FlowControlStrategy strategy)
{
if (strategy == null)
throw new ArgumentNullException("FlowControllerFactory.CreateFlowController.strategy"); IFlowController controller = null; switch (strategy.StrategyType)
{
case FlowControlStrategyType.TPS:
controller = new TPSFlowController(strategy);
break;
case FlowControlStrategyType.Delay:
controller = new DelayFlowController(strategy);
break;
case FlowControlStrategyType.Sum:
controller = new SumFlowController(strategy);
break;
default:
break;
} return controller;
}
}
}
Net Core的流控服务-LMLPHP

有了流控策略定义、我们更进一步,继续封装了流控Facade服务,这样把流控的变化封装到内部。对外只提供流控服务接口,流控时动态传入流控策略和流控个数:FlowControlService

Net Core的流控服务-LMLPHP
using System;
using System.Collections.Generic;
using System.Text; namespace CZ.FlowControl.Service
{
using CZ.FlowControl.Spi;
using System.Threading; /// <summary>
/// 统一流控服务
/// </summary>
public class FlowControlService
{
/// <summary>
/// 流控
/// </summary>
/// <param name="strategy">流控策略</param>
/// <param name="count">请求次数</param>
public static void FlowControl(FlowControlStrategy strategy, int count = 1)
{
var controller = FlowControllerFactory.GetInstance().GetOrCreateFlowController(strategy); TimeSpan waitTimespan = TimeSpan.Zero; var result = controller.ShouldThrottle(count, out waitTimespan);
if (result)
{
if (strategy.IsRefusedRequest == false && waitTimespan != TimeSpan.Zero)
{
WaitForAvailable(strategy, controller, waitTimespan, count);
}
else if (strategy.IsRefusedRequest)
{
throw new Exception("触发流控!");
}
}
} /// <summary>
/// 等待可用
/// </summary>
/// <param name="strategy">流控策略</param>
/// <param name="controller">流控器</param>
/// <param name="waitTimespan">等待时间</param>
/// <param name="count">请求次数</param>
private static void WaitForAvailable(FlowControlStrategy strategy, IFlowController controller, TimeSpan waitTimespan, int count)
{
var timespan = waitTimespan;
if (strategy.StrategyType == FlowControlStrategyType.Delay)
{
Thread.Sleep(timespan);
return;
} while (controller.ShouldThrottle(count, out timespan))
{
Thread.Sleep(timespan);
}
}
}
}
Net Core的流控服务-LMLPHP

以上,统一流控服务完成了第一个版本的封装。接下来我们看示例代码

 四、示例代码

    先安装Nuget:

Install-Package CZ.FlowControl.Service -Version 1.0.0

Net Core的流控服务-LMLPHP

Net Core的流控服务-LMLPHP

是不是很简单。

大家如果希望了解详细的代码,请参考这个项目的GitHub地址:

https://github.com/zhouguoqing/FlowControl

同时也欢迎大家一起改进完善。

周国庆

2019/8/9

 
05-08 15:51