在 WebRTC 中，如何标记本地 MediaStream 以便远程对等方可以识别它?

问题描述

我正在构建一个 WebRTC 应用程序，用户可以在其中共享他们的相机和屏幕.当客户端接收到一个流/轨迹时，它需要知道它是一个摄像头流还是一个屏幕录制流.这种区别在发送端很明显，但是当轨道到达接收端时，区别就消失了.

I am building a WebRTC application in which users can share their camera and their screen.When a client receives a stream/track, it needs to know whether it is a camera stream or a screen recording stream.This distinction is obvious at the sending end,but the distinction is lost by the time the tracks reach the receiving peer.

这是我的应用程序中的一些示例代码:

Here's some sample code from my application:

// Note the distinction between streams is obvious at the sending end.
const localWebcamStream = await navigator.mediaDevices.getUserMedia({ ... });
const screenCaptureStream = await navigator.mediaDevices.getDisplayMedia({ ... });

// This is called by signalling logic
function addLocalTracksToPeerConn(peerConn) {
  // Our approach here loses information because our two distinct streams
  // are added to the PeerConnection's homogeneous bag of streams

  for (const track of screenCaptureStream.getTracks()) {
    peerConn.addTrack(track, screenCaptureStream);
  }

  for (const track of localWebcamStream.getTracks()) {
    peerConn.addTrack(track, localWebcamStream);
  }
}

// This is called by signalling logic
function handleRemoteTracksFromPeerConn(peerConn) {
    peerConn.ontrack = ev => {
      const stream = ev.streams[0];
      if (stream is a camera stream) {  // FIXME how to distinguish reliably?
        remoteWebcamVideoEl.srcObject = stream;
      }
      else if (stream is a screen capture) {  // FIXME how to distinguish reliably?
        remoteScreenCaptureVideoEl.srcObject = stream;
      }
  };
}

我理想的虚构 API 将允许将 .label 添加到轨道或流中，如下所示:

My ideal imaginary API would allow adding a .label to a track or stream, like this:

// On sending end, add arbitrary metadata
track.label = "screenCapture";
peerConn.addTrack(track, screenCaptureStream);

// On receiving end, retrieve arbitrary metadata
peerConn.ontrack = ev => {
      const trackType = ev.track.label;  // get the label when receiving the track
}

但是这个API实际上并不存在.有 一个 MediaStreamTrack.label 属性,但它是只读的，并且不会在传输中保存.通过实验，发送端的 .label 属性提供信息(例如 label: FaceTime HD Camera (Built-in) (05ac:8514)").但在接收端，同一曲目的 .label 没有保留.(它似乎被曲目的 .id 替换了 - 至少在 Chrome 中是这样.)

But this API does not really exist.There is a MediaStreamTrack.label property,but it's read-only, and not preserved in transmission.By experimentation,the .label property at the sending end is informative (e.g. label: "FaceTime HD Camera (Built-in) (05ac:8514)").But at the receiving end, the .label for the same track is is not preserved.(It appears to be replaced with the .id of the track - in Chrome, at least.)

凯文·莫兰德的这篇文章描述了同样的问题，并推荐一个有点可怕的解决方案:在发送端修改 SDP，然后在接收端grep SDP.但是这个解决方案感觉非常脆弱和低级.

This article by Kevin Morelanddescribes the same problem,and recommends a mildly terrifying solution:munge the SDP on the sending end,and then grep the SDP on the receiving end.But this solution feels very fragile and low-level.

我知道有一个 MediaStreamTrack.id 属性.还有一个 MediaStream.id 属性.这两者似乎都在传输中得以保存.这意味着我可以在侧信道中发送元数据，例如信令通道或DataChannel.从发送端，我会发送 { "myStreams": { "screen": "", "camera": ""} }.在显示任何内容之前，接收端会等到它同时拥有元数据和流.但是，这种方法引入了侧信道(以及与之相关的不可避免的并发挑战)，不需要侧信道的地方.

I know there is a MediaStreamTrack.id property.There is also a MediaStream.id property.Both of these appear to be preserved in transmission.This means I could send the metadata in a side-channel,such as the signalling channel or a DataChannel.From the sending end, I would send { "myStreams": { "screen": "<some stream id>", "camera": "<another stream id>" } }.The receiving end would wait until it has both the metadata and the stream before displaying anything.However, this approach introduces a side-channel (and inevitable concurrency challenges associated with that),where a side-channel feels unnecessary.

我正在寻找一种惯用的、可靠的解决方案.如何在发送端标记/识别 MediaStreams，以便接收端知道哪个流是哪个?

I'm looking for an idiomatic, robust solution.How do I label/identify MediaStreams at the sending end,so that the receiving end knows which stream is which?

推荐答案

我最终在信令通道中发送了这个元数据.每个包含 SessionDescription (SDP) 的信令消息现在还包含元数据对象，它注释 SDP 中描述的 MediaStream.这没有并发问题，因为在为 MediaStream 触发 track 事件之前，客户端总是会收到 MediaStream 的 SDP+元数据.

I ended up sending this metadata in the signaling channel. Each signaling message that contained a SessionDescription (SDP) now also contains metadata object alongside it, which annotates the MediaStreams that are described in the SDP. This has no concurrency issues, because clients will always receive the SDP+metadata for a MediaStream before the track event is fired for that MediaStream.

所以以前我有这样的信令消息:

So previously I had signaling messages like this:

{
  "kind": "sessionDescription",

  // An RTCSessionDescriptionInit
  "sessionDescription": { "type": "offer", "sdp": "..." }
}

现在我有这样的信令消息:

Now I have signaling messages like this:

{
  "kind": "sessionDescription",

  // An RTCSessionDescriptionInit
  "sessionDescription": { "type": "offer", "sdp": "..." },

  // A map from MediaStream IDs to arbitrary domain-specific metadata
  "mediaStreamMetadata": {
    "y6w4u6e57654at3s5y43at4y5s46": { "type": "camera" },
    "ki8a3greu6e53a4s46uu7dtdjtyt": { "type": "screen" }
  }
}

这篇关于在 WebRTC 中，如何标记本地 MediaStream 以便远程对等方可以识别它?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！