使用云功能执行数据流模板时出错

问题描述

尝试使用Google Cloud函数执行自定义数据流模板时遇到错误.

Getting below error while trying to execute custom dataflow template using Google Cloud function.

错误:问题正在运行的数据流模板，错误是:{错误:尝试检索Compute Engine内置服务帐户的访问令牌时，返回了禁止的错误.这可能是因为Compute Engine实例没有指定了正确的权限范围.无法刷新访问令牌".

Error:"problem running dataflow template, error was: { Error: A Forbidden error was returned while attempting to retrieve an access token for the Compute Engine built-in service account. This may be because the Compute Engine instance does not have the correct permission scopes specified. Could not refresh access token".

我尝试提供所有必需的权限和范围.有人可以提出解决方案.

I have tried supplying all the required permissions and scopes.Could someone please suggest a resolution.

推荐答案

google-cloud 节点库尚不支持Dataflow API，因此当前使用该API的方法是 googleapis 库.

The google-cloud node library does not yet support the Dataflow API, so the current way to use that API is the googleapis library.

按照那里的说明，我尝试使用 Google提供的模板使用HTTP触发的功能，并且没有问题:

Following the instructions there, I've tried to launch a Dataflow job with a Google-provided template using an HTTP-triggered function, and had no issues:

const {google} = require('googleapis');
const project = "your-project-id"

exports.launchDataflowTemplate = (req, res) => {
    let result;
    google.auth.getApplicationDefault(function(err, authClient, projectId) {
            if (err) {
                throw err;
            }
            if (authClient.createScopedRequired && authClient.createScopedRequired()) {
                authClient = authClient.createScoped([
                    'https://www.googleapis.com/auth/cloud-platform',
                    'https://www.googleapis.com/auth/compute',
                    'https://www.googleapis.com/auth/compute.readonly',
                    'https://www.googleapis.com/auth/userinfo.email'
                ]);
            }
            var dataflow = google.dataflow({
                version: "v1b3",
                auth: authClient
            });

            var launchParams = {
                "inputFilePattern": "gs://your-input-bucket/*.gz",
                "outputDirectory": "gs://your-result-bucket/",
                "outputFailureFile": "gs://your-logs-bucket/error.csv"
            };

            var env = {
               "tempLocation": "gs://your-staging-bucket/temp",
               "zone": "us-central1-f"
            }


            var opts = {
                projectId: project,
                gcsPath: "gs://dataflow-templates/latest/Bulk_Decompress_GCS_Files",
                resource: {
                    parameters: launchParams,
                    environment: env
                }
            };

            dataflow.projects.templates.launch(opts, (err, result) => {
                if (err) {
                    throw err;
                }
                res.send(result.data);
            });
    });
};

这篇关于使用云功能执行数据流模板时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！