如何同步新的 ActiveStorage 镜像?

问题描述

从 ActiveStorage 开始，您可以知道定义用于存储文件的镜像.

Starting with ActiveStorage you can know define mirrors for storing your files.

local:
  service: Disk
  root: <%= Rails.root.join("storage") %>

amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: us-east-1
  bucket: mybucket

mirror:
  service: Mirror
  primary: local
  mirrors:
    - amazon
    - another_mirror

如果您在某个时间点后添加镜像，则必须注意复制所有文件，例如从本地"到亚马逊"或another_mirror".

If you add a mirror after a certain point of time you have to take care about copying all files e.g. from "local" to "amazon" or "another_mirror".

1. 是否有一种方便的方法来保持文件同步?
2. 或者方法运行验证以检查所有文件是否在每个服务上可用?

推荐答案

我有几个可能适合您的解决方案，一个适用于 Rails

I have a couple of solutions that might work for you, one for Rails <= 6.0 and one for Rails >= 6.1:

首先，您需要遍历 ActiveStorage Blob:

Firstly, you need to iterate through your ActiveStorage blobs:

ActiveStorage::Blob.all.each do |blob|
  # work with blob
end

然后...

1. Rails

您将需要 blob 的密钥、校验和以及磁盘上的本地文件.

You will need the blob's key, checksum, and the local file on disk.

local_file = ActiveStorage::Blob.service.primary.path_for blob.key

# I'm picking the first mirror as an example,
# but you can select a specific mirror if you want
mirror = blob.service.mirrors.first

mirror.upload blob.key, File.open(local_file), checksum: blob.checksum

如果镜像已存在，您可能还想避免上传文件.你可以这样做:

You may also want to avoid uploading a file if it already exists on the mirror. You can do that by doing this:

mirror = blob.service.mirrors.first

# If the file doesn't exist on the mirror, upload it
unless mirror.exist? blob.key
  # Upload file to mirror
end

把它放在一起，一个耙子任务可能看起来像:

Putting it together, a rake task might look like:

# lib/tasks/active_storage.rake

namespace :active_storage do

  desc 'Ensures all files are mirrored'
  task mirror_all: [:environment] do

  # Iterate through each blob
  ActiveStorage::Blob.all.each do |blob|

    # We assume the primary storage is local
    local_file = ActiveStorage::Blob.service.primary.path_for blob.key

    # Iterate through each mirror
    blob.service.mirrors.each do |mirror|

      # If the file doesn't exist on the mirror, upload it
      mirror.upload(blob.key, File.open(local_file), checksum: blob.checksum) unless mirror.exist? blob.key

      end
    end
  end
end

您可能会遇到类似提到的@Rystraum的情况，您可能需要从本地磁盘以外的其他地方进行镜像.在这种情况下，rake 任务可能如下所示:

You may run into a situation like @Rystraum mentioned where you might need to mirror from somewhere other than the local disk. In this case, the rake task could look like this:

# lib/tasks/active_storage.rake

namespace :active_storage do

  desc 'Ensures all files are mirrored'
  task mirror_all: [:environment] do

    # All services in our rails configuration
    all_services = [ActiveStorage::Blob.service.primary, *ActiveStorage::Blob.service.mirrors]

    # Iterate through each blob
    ActiveStorage::Blob.all.each do |blob|

      # Select services where file exists
      services = all_services.select { |file| file.exist? blob.key }

      # Skip blob if file doesn't exist anywhere
      next unless services.present?

      # Select services where file doesn't exist
      mirrors = all_services - services

      # Open the local file (if one exists)
      local_file = File.open(services.find{ |service| service.is_a? ActiveStorage::Service::DiskService }.path_for blob.key) if services.select{ |service| service.is_a? ActiveStorage::Service::DiskService }.any?

      # Upload local file to mirrors (if one exists)
      mirrors.each do |mirror|
        mirror.upload blob.key, local_file, checksum: blob.checksum
      end if local_file.present?

      # If no local file exists then download a remote file and upload it to the mirrors (thanks @Rystraum)
      services.first.open blob.key, checksum: blob.checksum do |temp_file|
        mirrors.each do |mirror|
          mirror.upload blob.key, temp_file, checksum: blob.checksum
        end
      end unless local_file.present?

    end
  end
end

虽然第一个 rake 任务回答了 OP 的问题，但后者的用途要广泛得多:

While the first rake task answers the OP's question, the latter is much more versatile:

• 它可以与任何服务组合一起使用
• 不需要 DiskService
• 优先通过 DiskServices 上传
• 避免额外的存在?调用，因为我们每个 blob 每个服务只调用一次

导轨 >6.1

它超级简单，只需在每个 blob 上调用它...

Its super easy, just call this on each blob...

blob.mirror_later

将其包装为 rake 任务如下所示:

Wrapping it up as a rake task looks like:

# lib/tasks/active_storage.rake

namespace :active_storage do

  desc 'Ensures all files are mirrored'
  task mirror_all: [:environment] do
    ActiveStorage::Blob.all.each do |blob|
      blob.mirror_later
    end
  end
end

这篇关于如何同步新的 ActiveStorage 镜像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！