流媒体文件，以S3和QUOT;错误：流意外＆QUOT结束;

问题描述

更新：

我相信这个的可以的是由我使用由前preSS提供的体解析器的事实造成的。难道这是玩弄的流多方试图解析？

I believe that this could be caused by the fact that I'm using the body parser provided by express. Could this be messing with the stream that multiparty is trying to parse?

我在我的基础上的解决方案这个答案。

I'm basing my solution on this answer.

我想要做的：流从客户端浏览器中的文件直接到S3和我的NodeJS服务器作为代理（出于安全目的）。我不希望文件通过触摸服务器的文件系统，以避免瓶颈。

What I'm trying to do: Stream a file from a client browser straight to S3 with my NodeJS server acting as a proxy (for security purposes). I don't want the file to touch the server's filesystem to avoid that bottleneck.

我收到以下错误：

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: stream ended unexpectedly
    at Form.<anonymous> (/Users/kentcdodds/Developer/bucketstreams/node_modules/multiparty/index.js:619:24)
    at Form.EventEmitter.emit (events.js:117:20)
    at finishMaybe (/Users/kentcdodds/Developer/bucketstreams/node_modules/multiparty/node_modules/readable-stream/lib/_stream_writable.js:443:14)
    at endWritable (/Users/kentcdodds/Developer/bucketstreams/node_modules/multiparty/node_modules/readable-stream/lib/_stream_writable.js:452:3)
    at Form.Writable.end (/Users/kentcdodds/Developer/bucketstreams/node_modules/multiparty/node_modules/readable-stream/lib/_stream_writable.js:419:5)
    at onend (_stream_readable.js:457:10)
    at process._tickCallback (node.js:415:13)

我看了看code和不能完全似乎明白了什么导致了问题。我使用角文件上传因为我采用了棱角分明的前端。下面是请求如下：

I've looked at the code and can't quite seem to understand what's causing the issue. I'm using angular-file-upload because I'm using angular on the front end. Here's what the request looks like:

Request URL:http://local.bucketstreams.com:3000/upload/image
Request Headers CAUTION: Provisional headers are shown.
Accept:application/json, text/plain, */*
Cache-Control:no-cache
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryNKuH2H9IUB7kvmea
Origin:http://local.bucketstreams.com:3000
Pragma:no-cache
Referer:http://local.bucketstreams.com:3000/getting-started
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36
Request Payload
------WebKitFormBoundaryNKuH2H9IUB7kvmea
Content-Disposition: form-data; name="type"

profile
------WebKitFormBoundaryNKuH2H9IUB7kvmea
Content-Disposition: form-data; name="user"

pie
------WebKitFormBoundaryNKuH2H9IUB7kvmea
Content-Disposition: form-data; name="file0"; filename="Screen Shot 2014-02-18 at 10.54.06 PM.png"
Content-Type: image/png


------WebKitFormBoundaryNKuH2H9IUB7kvmea--

和这里是我的code是这样的：

And here's what my code looks like:

var ErrorController = require('../controller/ErrorController');
var AuthenticationController = require('../controller/AuthenticationController');
var logger = require('winston');

var http = require('http');
var util = require('util');
var multiparty = require('multiparty');
var knox = require('knox');
var Batch = require('batch');

var s3Client = knox.createClient({
  secure: false,
  key: process.env.S3_KEY,
  secret: process.env.S3_SECRET,
  bucket: process.env.S3_BUCKET_IMAGES
});

var Writable = require('readable-stream').Writable;
util.inherits(ByteCounter, Writable);
function ByteCounter(options) {
  Writable.call(this, options);
  this.bytes = 0;
}

ByteCounter.prototype._write = function(chunk, encoding, cb) {
  this.bytes += chunk.length;
  cb();
};

var supportedTypes = {
  profile: true,
  post: true
};

module.exports = function(app) {
  app.post('/upload/image', AuthenticationController.checkAuthenticated, function(req, res) {
    var type = req.body.type;
    var userId = req.user._id;
    if (!supportedTypes[type]) {
      return ErrorController.sendErrorJson(res, 401, 'Unsupported image upload type: ' + type);
    }

    var headers = {
      'x-amz-acl': 'public-read'
    };
    var form = new multiparty.Form();
    var batch = new Batch();
    batch.push(function(cb) {
      form.on('field', function(name, value) {
        if (name === 'path') {
          var destPath = value;
          if (destPath[0] !== '/') destPath = '/' + destPath;
          cb(null, destPath);
        }
      });
    });

    batch.push(function(cb) {
      form.on('part', function(part) {
        if (! part.filename) return;
        cb(null, part);
      });
    });

    batch.end(function(err, results) {
      if (err) throw err;
      form.removeListener('close', onEnd);
      var destPath = '/' + userId + results[0];
      var part = results[1];

      var counter = new ByteCounter();
      part.pipe(counter); // need this until knox upgrades to streams2
      headers['Content-Length'] = part.byteCount;
      s3Client.putStream(part, destPath, headers, function(err, s3Response) {
        if (err) throw err;
        res.statusCode = s3Response.statusCode;
        s3Response.pipe(res);
        console.log('https://s3.amazonaws.com/' + process.env.S3_BUCKET_IMAGES + destPath);
      });
      part.on('end', function() {
        console.log('part end');
        console.log('size', counter.bytes);
      });
    });

    form.on('close', function(error) {
      logger.error(error);
      return ErrorController.sendErrorJson(res, 500, 'There was a problem uploading the file.');
    });
    form.parse(req);
  });
};

它看起来像被吹起来的部分是多方，我已经查看了codeA有点不得要领。我不能肯定，如果我提出请求的不正确，或者如果有什么问题我的服务器code。我不认为这有什么与我的S3存储桶，但我想这可能是也。

It looks like the part that is blowing up is multiparty and I've looked into that code a little bit to no avail. I'm not certain if I'm making the request incorrectly or if there's something wrong with my server code. I don't think it has anything to do with my S3 bucket, but I suppose it could be that as well.

总之，任何技巧都是值得欢迎的。

Anyway, any tips are welcome.

推荐答案

症结的解决方法是不使用（显然是去precated） bodyParser（）。我不能确定它做什么，但它搞砸了窗体的部件多方使用。所以，相反，如果你有同样的问题我有，而不是使用 bodyParser（），使用你需要的东西明确地（例如）：

The crux to the solution is to not use the (apparently deprecated) bodyParser(). I'm not certain what it does, but it screws up the form's parts that multiparty uses. So instead, if you have the same problem I had, instead of using bodyParser(), use the things you need explicitely (for example):

app.use(express.urlencoded());
app.use(express.json());

然后为多部分的东西，只是用多方给自己解析身体。多方的作者给出更多信息关于这个问题的。

这篇关于流媒体文件，以S3和QUOT;错误：流意外＆QUOT结束;的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！