带有嵌入式 Bokeh 服务器应用程序的 Flask 中的代码 503 通过 requests.get() 获取 jsonified 数据

问题描述

我正在参数化我的散景应用程序，方法是让我的 Flask 应用程序通过一个专门用于 jsonifying 通过查询字符串参数传递的请求数据的路由公开模型数据.我知道数据发送路由有效，因为当我将它用作 AjaxDataSource 的 url 时，我得到了预期的数据.然而，当我尝试使用 requests.get api 进行等效操作时，我得到一个 503 响应代码，这让我觉得我在这里违反了一些基本的东西，我无法完全掌握我有限的 webdev 经验.我做错了什么或违反了什么?

实际上，我需要比 AjaxDataSource 提供的列限制更多的数据检索灵活性.我希望依靠 requests 模块通过序列化和反序列化 Json 来传递任意类实例和其他东西.

这是我演示从

这是一些调试输出...

C:TestApp>python flask_embedJSONRoute.py在 http://localhost:8080/上打开带有嵌入式 Bokeh 应用程序的 Flask 应用程序>C:TestAppflask_embedjsonroute.py(52)modify_doc2()->res = requests.get( url , timeout=None , verify=False )(Pdb) n>C:TestAppflask_embedjsonroute.py(53)modify_doc2()->打印(代码 %s" % res.status_code )(Pdb) n代码 503>C:TestAppflask_embedjsonroute.py(54)modify_doc2()->打印(编码 %s" % res.encoding )(Pdb) n编码 utf-8>C:TestAppflask_embedjsonroute.py(55)modify_doc2()->打印(文本 %s" % res.text )(Pdb) n文本>C:TestAppflask_embedjsonroute.py(56)modify_doc2()->数据 = res.json()(Pdb)文件C:Anaconda3libjsondecoder.py"，第 357 行，raw_decode从 None 提高 JSONDecodeError("Expecting value", s, err.value)json.decoder.JSONDecodeError:期望值:第 1 行第 1 列(字符 0)

解决方案

这似乎不是 Bokeh 本身的问题，而是运行 Flask 应用程序的服务器中的线程和阻塞问题.

除了散景之外，它完全可以重现......

导入请求从烧瓶导入烧瓶，jsonify，请求进口大熊猫导入 pdbflask_app = Flask(__name__)# 填充由烧瓶应用程序维护的一些模型modelDf = pandas.DataFrame()数据 = 100modelDf['c1_x'] = 范围(nData)modelDf['c1_y'] = [x*x for x in range(nData)]modelDf['c2_x'] = 范围(nData)modelDf['c2_y'] = [2*x for x in range(nData)]@flask_app.route('/', methods=['GET'])定义索引():res = ""res += "<tr><td><a href="http://localhost:8080/sendModelData/c1">SEND C1</a></td></tr>"res += "<tr><td><a href="http://localhost:8080/sendModelData/c2">SEND C2</a></td></tr>"res += "<tr><td><a href="http://localhost:8080/RequestsOverFlaskNoProxy?colName=c1">REQUEST OVER FLASK NO PROXY C1</a></td></tr>"res += "<tr><td><a href="http://localhost:8080/RequestsOverFlaskNoProxy?colName=c2">REQUEST OVER FLASK NO PROXY C2</a></td></tr>"res += "<tr><td><a href="http://localhost:8080/RequestsOverFlask?colName=c1">REQUEST OVER FLASK C1</a></td><;/tr>"res += "<tr><td><a href="http://localhost:8080/RequestsOverFlask?colName=c2">REQUEST OVER FLASK C2</a></td><;/tr>"res += "</table>"返回资源@flask_app.route('/RequestsOverFlaskNoProxy')def requestsOverFlaskNoProxy() :打印(RequestsOverFlaskNoProxy")# 从查询字符串中获取列名colName = request.args.get('colName')# 从 Flask 获取模型数据url = "http://localhost:8080/sendModelData/%s" % colNameprint("从 %s 获取数据" % url )session = requests.Session()session.trust_env = Falseres = session.get( url , timeout=5000 , verify=False )打印(代码 %s" % res.status_code )打印(编码 %s" % res.encoding )打印(文本 %s" % res.text )数据 = res.json()返回数据@flask_app.route('/RequestsOverFlask')def requestsOverFlask() :# 从查询字符串中获取列名colName = request.args.get('colName')# 从 Flask 获取模型数据url = "http://localhost:8080/sendModelData/%s" % colNameres = requests.get( url , timeout=None , verify=False )打印(代码 %s" % res.status_code )打印(编码 %s" % res.encoding )打印(文本 %s" % res.text )数据 = res.json()返回数据@flask_app.route('/sendModelData/' , methods=['GET'] )def sendModelData( colName ) :x = modelDf[ colName + "_x" ].tolist()y = modelDf[ colName + "_y" ].tolist()返回 jsonify( x=x , y=y )如果 __name__ == '__main__':print('在 http://localhost:8080/上打开 Flask 应用程序')# 这不起作用#flask_app.run( host='0.0.0.0' , port=8080 , debug=True )# 这有效flask_app.run( host='0.0.0.0' , port=8080 , debug=True , threaded=True )

从屏幕截图中可以看出，直接从 sendModelData 提供数据会适当地呈现 JSon，但是当通过 requests.get 方法获取时，由于Python 控制台中报告的 503 代码.

如果我同样尝试消除 代理，我已通过环境变量启用，但这种方法永远不会完成，并且请求使浏览器无限期地旋转.

想想看，甚至将请求用作中间人也可能完全没有必要，我应该能够只获取 json 字符串并自己进行反序列化.好吧，这将在我的实际代码中在此设置中起作用，散景渲染是在与 Flask 应用程序完全不同的 Python 模块中完成的，因此除非我扰乱应用程序的分层，否则这些功能甚至不可用.

编辑事实证明，我违反了 Flask 的开发环境...

您正在使用 Flask 测试服务器运行您的 WSGI 应用程序，它由默认使用单个线程来处理请求.所以当你的请求线程试图回调到同一个服务器，它仍然是忙着处理那个请求.https://stackoverflow.com/a/22878916/1330381

那么问题就变成了如何在原始散景示例中应用这种 threaded=True 技术?这可能是不可能的，flask_embed.py 示例对 Tornado WSGI 服务器的依赖来自这个 问题表明 Tornado 是单线程设计的.鉴于上述发现，一个更尖锐的问题是 AjaxDataSource 如何一起避免 requests 模块面临的这些线程问题?

更新关于散景和龙卷风耦合的更多背景...

53:05 所以他们实际上并不是很多，问题是关于Bokeh 和 Bokeh 服务器的依赖项.新的散景服务器是建立在龙卷风上，这几乎是它使用的主要依赖项龙卷风.除此之外，没有太多依赖项，运行时依赖项，用于散景.pandas 是一个可选的依赖项散景图.还有其他依赖项，您知道使用了 numpy.但只有，我认为依赖项列表有六到七个.我们已经多年来试图大幅削减它，但主要的服务器的依赖是龙卷风.数据可视化简介散景 - 第 1 部分 - Strata Hadoop San Jose2016

I'm in the process of parameterizing my bokeh apps by having my Flask app expose model data via a route dedicated to jsonifying the requested data passed via query string arguments. I know the data sending route works since when I use it as a url to AjaxDataSource I get the expected data plotted. However when I attempt the equivalent operation using the requests.get api I get a 503 response code which makes me think I'm violating something fundamental here I can't quite grasp with my limited webdev experience. What am I doing wrong and or violating?

I actually need a bit more data retrieving flexibility than the AjaxDataSource provides with its columnar limitations. I was hoping to lean on the requests module to pass arbitrary class instances and what not around by serializing and deserializing Json.

Here's the minimal example I have demonstrating the failure derived from flask_embed.html...

import requests
from flask import Flask, jsonify, render_template
import pandas
from tornado.ioloop import IOLoop

from bokeh.application          import Application
from bokeh.application.handlers import FunctionHandler
from bokeh.embed                import server_document
from bokeh.layouts              import column
from bokeh.models               import AjaxDataSource,ColumnDataSource
from bokeh.plotting             import figure
from bokeh.server.server        import Server

flask_app = Flask(__name__)

# Populate some model maintained by the flask application
modelDf = pandas.DataFrame()
nData = 100
modelDf[ 'c1_x' ] = range(nData)
modelDf[ 'c1_y' ] = [ x*x for x in range(nData) ]
modelDf[ 'c2_x' ] = range(nData)
modelDf[ 'c2_y' ] = [ 2*x for x in range(nData) ]

def modify_doc1(doc):
    # get colum name from query string
    args      = doc.session_context.request.arguments
    paramName = str( args['colName'][0].decode('utf-8') )

    # get model data from Flask
    url    = "http://localhost:8080/sendModelData/%s" % paramName
    source = AjaxDataSource( data             = dict( x=[] , y=[] ) ,
                            data_url         = url       ,
                            polling_interval = 5000      ,
                            mode             = 'replace' ,
                            method           = 'GET'     )
    # plot the model data
    plot = figure( )
    plot.circle( 'x' , 'y' , source=source , size=2 )
    doc.add_root(column(plot))

def modify_doc2(doc):
    # get column name from query string
    args    = doc.session_context.request.arguments
    colName = str( args['colName'][0].decode('utf-8') )

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName
    #pdb.set_trace()
    res = requests.get( url , timeout=None , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()

    # plot the model data
    plot = figure()
    plot.circle( 'x' , 'y' , source=data , size=2 )
    doc.add_root(column(plot))


bokeh_app1 = Application(FunctionHandler(modify_doc1))
bokeh_app2 = Application(FunctionHandler(modify_doc2))

io_loop = IOLoop.current()

server = Server({'/bkapp1': bokeh_app1 , '/bkapp2' : bokeh_app2 }, io_loop=io_loop, allow_websocket_origin=["localhost:8080"])
server.start()

@flask_app.route('/', methods=['GET'] )
def index():
    res =  "<table>"
    res += "<tr><td><a href="http://localhost:8080/app1/c1">APP1 C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/app1/c2">APP1 C2</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/app2/c1">APP2 C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/app2/c2">APP2 C2</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/sendModelData/c1">DATA C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/sendModelData/c2">DATA C2</a></td></tr>"
    res += "</table>"
    return res

@flask_app.route( '/app1/<colName>' , methods=['GET'] )
def bkapp1_page( colName ) :
    script = server_document( url='http://localhost:5006/bkapp1' , arguments={'colName' : colName } )
    return render_template("embed.html", script=script)

@flask_app.route( '/app2/<colName>' , methods=['GET'] )
def bkapp2_page( colName ) :
    script = server_document( url='http://localhost:5006/bkapp2', arguments={'colName' : colName } )
    return render_template("embed.html", script=script)

@flask_app.route('/sendModelData/<colName>' , methods=['GET'] )
def sendModelData( colName ) :
    x = modelDf[ colName + "_x" ].tolist()
    y = modelDf[ colName + "_y" ].tolist()
    return jsonify( x=x , y=y )

if __name__ == '__main__':
    from tornado.httpserver import HTTPServer
    from tornado.wsgi import WSGIContainer
    from bokeh.util.browser import view

    print('Opening Flask app with embedded Bokeh application on http://localhost:8080/')

    # This uses Tornado to server the WSGI app that flask provides. Presumably the IOLoop
    # could also be started in a thread, and Flask could server its own app directly
    http_server = HTTPServer(WSGIContainer(flask_app))
    http_server.listen(8080)

    io_loop.add_callback(view, "http://localhost:8080/")
    io_loop.start()

Here's the pages rendered...

Here's some debug output...

C:TestApp>python flask_embedJSONRoute.py
Opening Flask app with embedded Bokeh application on http://localhost:8080/
> C:TestAppflask_embedjsonroute.py(52)modify_doc2()
-> res = requests.get( url , timeout=None , verify=False )
(Pdb) n
> C:TestAppflask_embedjsonroute.py(53)modify_doc2()
-> print( "CODE %s" % res.status_code )
(Pdb) n
CODE 503
> C:TestAppflask_embedjsonroute.py(54)modify_doc2()
-> print( "ENCODING %s" % res.encoding )
(Pdb) n
ENCODING utf-8
> C:TestAppflask_embedjsonroute.py(55)modify_doc2()
-> print( "TEXT %s" % res.text )
(Pdb) n
TEXT
> C:TestAppflask_embedjsonroute.py(56)modify_doc2()
-> data = res.json()
(Pdb)

  File "C:Anaconda3libjsondecoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

解决方案

This appears to be not an issue with Bokeh per se but rather an issue with threading and blocking in the server that's running the Flask app.

It's reproducible apart from Bokeh entirely...

import requests
from flask import Flask, jsonify, request
import pandas
import pdb

flask_app = Flask(__name__)

# Populate some model maintained by the flask application
modelDf = pandas.DataFrame()
nData = 100
modelDf[ 'c1_x' ] = range(nData)
modelDf[ 'c1_y' ] = [ x*x for x in range(nData) ]
modelDf[ 'c2_x' ] = range(nData)
modelDf[ 'c2_y' ] = [ 2*x for x in range(nData) ]

@flask_app.route('/', methods=['GET'] )
def index():
    res =  "<table>"
    res += "<tr><td><a href="http://localhost:8080/sendModelData/c1">SEND C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/sendModelData/c2">SEND C2</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/RequestsOverFlaskNoProxy?colName=c1">REQUEST OVER FLASK NO PROXY C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/RequestsOverFlaskNoProxy?colName=c2">REQUEST OVER FLASK NO PROXY C2</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/RequestsOverFlask?colName=c1">REQUEST OVER FLASK C1</a></td></tr>"
    res += "<tr><td><a href="http://localhost:8080/RequestsOverFlask?colName=c2">REQUEST OVER FLASK C2</a></td></tr>"
    res += "</table>"
    return res

@flask_app.route('/RequestsOverFlaskNoProxy')
def requestsOverFlaskNoProxy() :
    print("RequestsOverFlaskNoProxy")
    # get column name from query string
    colName = request.args.get('colName')

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName

    print("Get data from %s" % url )
    session = requests.Session()
    session.trust_env = False
    res = session.get( url , timeout=5000 , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()
    return data

@flask_app.route('/RequestsOverFlask')
def requestsOverFlask() :
    # get column name from query string
    colName = request.args.get('colName')

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName
    res = requests.get( url , timeout=None , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()
    return data

@flask_app.route('/sendModelData/<colName>' , methods=['GET'] )
def sendModelData( colName ) :
    x = modelDf[ colName + "_x" ].tolist()
    y = modelDf[ colName + "_y" ].tolist()
    return jsonify( x=x , y=y )

if __name__ == '__main__':
    print('Opening Flask app on http://localhost:8080/')

    # THIS DOES NOT WORK
    #flask_app.run( host='0.0.0.0' , port=8080 , debug=True )

    # THIS WORKS
    flask_app.run( host='0.0.0.0' , port=8080 , debug=True , threaded=True )

One can see from the screen shot that serving data directly from sendModelData renders the JSon appropriately, but when fetched via the requests.get method yields an exception due to a 503 code as reported in the Python Console.

If I make the same attempt trying to eliminate the effect of the proxies which I have enabled via environment variables but this approach never completes and the request leaves the browser spinning indefinitely.

Come to think of it it may be completely unnecessary to even use requests as a middle man and I should be able to just get the json string and go about deserializing it myself. Well, that would work in this setup by in my actual code the Bokeh rendering is done in a completely different python Module than the Flask application so these functions are not even available unless I scramble the layering of the app.

EDITAs it turns out the fundamental thing I was violating was with Flask's development environment...

So then the question becomes how to apply this threaded=True technique in the original Bokeh example? This may not be possible by the flask_embed.py example's reliance on the Tornado WSGI server which from this question suggests Tornado is single threaded by design.Given the above findings an even keener question is how does the AjaxDataSource all together avoid these threading issues faced by the requests module?

---

UpdateSome more background on the Bokeh and Tornado coupling...

这篇关于带有嵌入式 Bokeh 服务器应用程序的 Flask 中的代码 503 通过 requests.get() 获取 jsonified 数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！