尝试使用 TFjs 和 React Native 进行实时对象检测，始终给出相同的预测并在打开相机时停止

问题描述

当相机打开时，一个空白的相机会出现几秒钟，它总是给出相同的输出并停止.

When the camera opens a blank camera appears for a few seconds and it always gives the same below output and stops.

预测:[{className":线虫，线虫蠕虫，蛔虫"，概率":0.050750732421875}，{className":火柴棒"，概率.46"359:30},{className":打火机，灯光，点火器，点火器"，概率":0.021453857421875}]

知道如何进行实时预测吗?没有得到一次如上的错误预测

Any idea how I can make the real time prediction work? without getting a false prediction as above just for one time

以下是相机屏幕代码，当用户扫描某个环境时，预测应该在实时相机馈送中发生

Below is the Camera Screen code where the prediction should happen in real time camera feed when user scans a certain surrounding

export function CameraScreen() {
  const [word, setWord] = useState("");
  const [predictionFound, setPredictionFound] = useState(false);
  const [hasPermission, setHasPermission] = useState();
  const [model, setModel] = useState();
  const TensorCamera = cameraWithTensors(Camera);
  let requestAnimationFrameId = 0;
  const textureDims =
    Platform.OS === "ios"
      ? { width: 1080, height: 1920 }
      : { width: 1600, height: 1200 };
  const tensorDims = { width: 152, height: 200 };

  async function loadModel() {
    try {
      const model = await mobilenet.load();
      setModel(model);
      console.log("set loaded Model");
    } catch (err) {
      console.log(err);
      console.log("failed load model");
    }
  }

  useEffect(() => {
    (async () => {
      const { status } = await Camera.requestPermissionsAsync();
      setHasPermission(status === "granted");

      await tf.ready();
      await loadModel();
      console.log("after model load");
    })();
  }, []);

  const getPrediction = async (tensor) => {
    if (!tensor) {
      return;
    }

    const prediction = await model.classify(tensor);
    console.log(`prediction: ${JSON.stringify(prediction)}`);

    if (!prediction || prediction.length === 0) {
      cancelAnimationFrame(requestAnimationFrameId);
      console.log("no predictions found");
      setPredictionFound(false);
      return;
    } else {
      setPredictionFound(true);
    }
  };

  const handleCameraStream = (imageAsTensors) => {
    const loop = async () => {
      const nextImageTensor = await imageAsTensors.next().value;
      await getPrediction(nextImageTensor);
      requestAnimationFrameId = requestAnimationFrame(loop);
    };
    if (!predictionFound) {
      loop();
    }
  };

  const renderCameraView = () => {
    return (
      <View style={styles.cameraView}>
        <TensorCamera
          style={styles.camera}
          type={Camera.Constants.Type.back}
          zoom={0}
          cameraTextureHeight={textureDims.height}
          cameraTextureWidth={textureDims.width}
          resizeHeight={tensorDims.height}
          resizeWidth={tensorDims.width}
          resizeDepth={3}
          onReady={handleCameraStream}
        />
      </View>
    );
  };

  useEffect(() => {
    return () => {
      cancelAnimationFrame(requestAnimationFrameId);
    };
  }, [requestAnimationFrameId]);

  return (
    <View style={styles.container}>
      <View style={styles.header}>
        <Text style={styles.title}>My Pictionary</Text>
      </View>
      {model ? (
        <View style={styles.body}>{renderCameraView()}</View>
      ) : (
        <Text>Still loading</Text>
      )}
    </View>
  );
}

推荐答案

在函数 handleCameraStream 中，一旦找到预测，您就停止循环该函数.在您的情况下，您希望不断运行循环，因为您希望对所有帧而不是单个帧进行预测.

In the function handleCameraStream you stop looping the function once a prediction is found. In your case you would want to constantly run the loop as you want to make predictions on all the frames not a single one.

const handleCameraStream = (imageAsTensors) => {
    const loop = async () => {
      const nextImageTensor = await imageAsTensors.next().value;
      await getPrediction(nextImageTensor);
      requestAnimationFrameId = requestAnimationFrame(loop);
    };
    loop();
  };

这篇关于尝试使用 TFjs 和 React Native 进行实时对象检测，始终给出相同的预测并在打开相机时停止的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！