问题描述
我正在尝试使用 Python 通过 HTTP 实现诸如tail -f"之类的东西.目前,我正在尝试使用 Tornado,但它一次只处理一个连接,即使我执行异步请求也是如此.
导入套接字导入子流程导入 tornado.gen 作为 gen导入 tornado.httpserver导入 tornado.ioloop导入 tornado.iostream导入 tornado.options导入 tornado.web从 tornado.options 导入定义,选项定义(端口",默认值=8888,帮助=在给定端口上运行",类型=整数)定义(输入文件",默认=test.txt",help="我们将'tail'的文件的路径",类型=str)类 MainHandler(tornado.web.RequestHandler):@tornado.web.asynchronous@gen.engine定义获取(自我):打印得到请求"输入文件 = 打开(选项.输入文件)p = subprocess.Popen("./nettail.py",标准输入=输入文件,标准输出=子进程.PIPE)port_number = int(p.stdout.readline().strip())self.write("
")self.write("你好,世界\n")self.flush()s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)流 = tornado.iostream.IOStream(s)产量 gen.Task(stream.connect, ("127.0.0.1", port_number))为真:数据 = 产量 gen.Task(stream.read_until, "\n")self.write(数据)self.flush()定义主():tornado.options.parse_command_line()application = tornado.web.Application([(r"/", MainHandler),])http_server = tornado.httpserver.HTTPServer(application)http_server.listen(options.port)tornado.ioloop.IOLoop.instance().start()如果 __name__ == "__main__":主要的()
我开始的过程是一个简单的尾巴",它输出到一个套接字.
随机导入进口插座导入系统导入时间#创建一个 INET、流媒体套接字s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)# 打开连接.尝试:对于 xrange(5) 中的尝试编号:port_number = random.randint(9000, 65000)尝试:s.bind(("localhost", port_number))除了socket.error:继续# 我们绑定成功了!sys.stdout.write("{0}".format(port_number))sys.stdout.write("\n")sys.stdout.flush()休息#成为服务器套接字s.听(5)# 接受一个连接.尝试:(clientsocket, 地址) = s.accept()为真:行 = sys.stdin.readline()如果不是行:时间.睡眠(1)继续clientsocket.sendall(行)最后:clientsocket.close()最后:s.close()
./nettail.py 工作正常,但 Tornado HTTP 服务器一次只处理一个请求.
我想使用长时间运行的持久 HTTP 连接来执行此操作,因为它与旧浏览器兼容.我知道 Web Sockets 是在现代浏览器中实现的方式.
我在 Linux 和 Solaris 上运行它,而不是 Windows.这意味着我可以在文件上使用 tornado.iostream,而不是通过套接字程序.尽管如此,这不是一个记录在案的特性,所以我为每个连接启动了一个套接字程序.
解决方案经过多次调试,发现这个尾服务器并没有阻塞.
我试图在打开两个 Firefox 窗口的情况下测试并发连接,但 Firefox 在第一个窗口被手动停止之前不会开始获取第二个窗口.我猜 Firefox 不喜欢有两个并发的 HTTP 连接来获取相同的资源.
打开 Firefox 窗口和 Chromium 窗口,我可以看到推送到两个选项卡的tail"输出.
谢谢大家的帮助.@abernert 的评论特别有帮助.
在即将发布的 Tornado 2.4.2 版本中,一个 "Pipe" IOStream 已实现.使用这个和常规的尾巴"大大简化了代码.
导入子流程导入 tornado.httpserver导入 tornado.ioloop导入 tornado.iostream导入 tornado.options导入 tornado.web从 tornado.options 导入定义,选项定义(端口",默认值=8888,帮助=在给定端口上运行",类型=整数)定义(输入文件",默认=test.txt",help="我们将'tail'的文件的路径",类型=str)类 MainHandler(tornado.web.RequestHandler):@tornado.web.asynchronous定义获取(自我):打印得到请求"self.p = subprocess.Popen(["tail", "-f", options.inputfile, "-n+1"],标准输出=子进程.PIPE)self.write("
")self.write("你好,世界\n")self.flush()self.stream = tornado.iostream.PipeIOStream(self.p.stdout.fileno())self.stream.read_until("\n", self.line_from_nettail)def on_connection_close(self, *args, **kwargs):"""连接关闭时清理nettail进程."""打印连接已关闭!!!!"self.p.terminate()tornado.web.RequestHandler.on_connection_close(self, *args, **kwargs)def line_from_nettail(self, data):self.write(数据)self.flush()self.stream.read_until("\n", self.line_from_nettail)定义主():tornado.options.parse_command_line()application = tornado.web.Application([(r"/", MainHandler),])http_server = tornado.httpserver.HTTPServer(application)http_server.listen(options.port)tornado.ioloop.IOLoop.instance().start()如果 __name__ == "__main__":主要的()
I'm trying to implement something like "tail -f" over HTTP with Python. Currently, I'm trying to use Tornado, but it only is handling one connection at a time, even when I do asynchronous requests.
import socket import subprocess import tornado.gen as gen import tornado.httpserver import tornado.ioloop import tornado.iostream import tornado.options import tornado.web from tornado.options import define, options define("port", default=8888, help="run on the given port", type=int) define( "inputfile", default="test.txt", help="the path to the file which we will 'tail'", type=str) class MainHandler(tornado.web.RequestHandler): @tornado.web.asynchronous @gen.engine def get(self): print "GOT REQUEST" inputfile = open(options.inputfile) p = subprocess.Popen( "./nettail.py", stdin=inputfile, stdout=subprocess.PIPE) port_number = int(p.stdout.readline().strip()) self.write("<pre>") self.write("Hello, world\n") self.flush() s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) stream = tornado.iostream.IOStream(s) yield gen.Task(stream.connect, ("127.0.0.1", port_number)) while True: data = yield gen.Task(stream.read_until, "\n") self.write(data) self.flush() def main(): tornado.options.parse_command_line() application = tornado.web.Application([ (r"/", MainHandler), ]) http_server = tornado.httpserver.HTTPServer(application) http_server.listen(options.port) tornado.ioloop.IOLoop.instance().start() if __name__ == "__main__": main()
The process I am starting is a simple "tail" which outputs to a socket.
import random import socket import sys import time #create an INET, STREAMing socket s = socket.socket( socket.AF_INET, socket.SOCK_STREAM) # Open the connection. try: for attempt_number in xrange(5): port_number = random.randint(9000, 65000) try: s.bind(("localhost", port_number)) except socket.error: continue # We successfully bound! sys.stdout.write("{0}".format(port_number)) sys.stdout.write("\n") sys.stdout.flush() break #become a server socket s.listen(5) # Accept a connection. try: (clientsocket, address) = s.accept() while True: line = sys.stdin.readline() if not line: time.sleep(1) continue clientsocket.sendall(line) finally: clientsocket.close() finally: s.close()
./nettail.py works as I expect, but the Tornado HTTP server is only handling one request at a time.
I would like to use long-running, persistent HTTP connections to do this, as it is compatible with older browsers. I understand that Web Sockets is how it would be done in modern browsers.
Edit:I'm running this on Linux and Solaris, not Windows. That means I could use tornado.iostream on the file, rather than through a sockets program. Still, that is not a documented feature, so I launch a sockets program for each connection.
解决方案After doing some more debugging, it turns out that this tail server was not blocking, after all.
I was trying to test concurrent connections with two windows of Firefox open, but Firefox would not start fetching the second window until the first window was manually stopped. I guess Firefox does not like to have two concurrent HTTP connections to fetch the same resource.
Opening a Firefox window and a Chromium window, I can see the "tail" output pushed to both tabs.
Thank you for all your help. @abarnert's comments were especially helpful.
Edit:
In the to-be-release 2.4.2 version of Tornado, a "Pipe" IOStream is implemented. Using this and regular "tail" simplified the code a lot.
import subprocess import tornado.httpserver import tornado.ioloop import tornado.iostream import tornado.options import tornado.web from tornado.options import define, options define("port", default=8888, help="run on the given port", type=int) define( "inputfile", default="test.txt", help="the path to the file which we will 'tail'", type=str) class MainHandler(tornado.web.RequestHandler): @tornado.web.asynchronous def get(self): print "GOT REQUEST" self.p = subprocess.Popen( ["tail", "-f", options.inputfile, "-n+1"], stdout=subprocess.PIPE) self.write("<pre>") self.write("Hello, world\n") self.flush() self.stream = tornado.iostream.PipeIOStream(self.p.stdout.fileno()) self.stream.read_until("\n", self.line_from_nettail) def on_connection_close(self, *args, **kwargs): """Clean up the nettail process when the connection is closed. """ print "CONNECTION CLOSED!!!!" self.p.terminate() tornado.web.RequestHandler.on_connection_close(self, *args, **kwargs) def line_from_nettail(self, data): self.write(data) self.flush() self.stream.read_until("\n", self.line_from_nettail) def main(): tornado.options.parse_command_line() application = tornado.web.Application([ (r"/", MainHandler), ]) http_server = tornado.httpserver.HTTPServer(application) http_server.listen(options.port) tornado.ioloop.IOLoop.instance().start() if __name__ == "__main__": main()
这篇关于我将如何实现“尾巴"?通过 HTTP 使用 Python Tornado?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!