S3

S3

关注
发信
关注(28)粉丝(399)

使用s3-sftp-proxy 暴露minio s3 数据为sftp 访问

尽管s3 很不错，但是ftp 也有自己存在的价值，以下是一个简单的通过s3-sftp-proxy 暴露minio s3 数据为ftp 的访问方式

环境准备

  • docker-compose 文件
version: "3"
services: 
  minio:
    image: minio/minio
    command: server /export
    ports:
    - "9000:9000"
    environment:
    - "MINIO_ACCESS_KEY=dalongdemo"
    - "MINIO_SECRET_KEY=dalongdemo"
  s3-proxy:
    build: ./
    image: dalongrong/s3-sftp-proxy
    command: -debug
    ports: 
    - "10022:10022"
    volumes: 
    - "./host_key:/app/host_key"
    - "./s3-sftp-proxy.example.toml:/app/s3-sftp-proxy.example.toml"
 
git clone https://github.com/moriyoshi/s3-sftp-proxy.git
cd s3-sftp-proxy 
go mod init 
go mod tidy
 

本地构建

go  build

docker 镜像通过多阶段构建

FROM golang:alpine as build
WORKDIR /app
ENV GOPROXY=https://mirrors.aliyun.com/goproxy/
RUN apk update && apk add wget unzip build-base git bzr mercurial gcc 
RUN git clone https://github.com/moriyoshi/s3-sftp-proxy.git
RUN cd s3-sftp-proxy && go mod init && go mod tidy && go build 
​
FROM alpine:latest
LABEL EMAIL="[email protected]"
LABEL AUTHOR="dalongrong"
WORKDIR /app
ENV BIND=:10022
ENV CONFIG=/app/s3-sftp-proxy.example.toml
EXPOSE 10022
COPY entrypoint.sh /app/entrypoint.sh
RUN chmod +x /app/entrypoint.sh
ENV PATH=$PATH:/usr/local/bin
COPY s3-sftp-proxy.example.toml /app/s3-sftp-proxy.example.toml
COPY --from=build /app/s3-sftp-proxy/s3-sftp-proxy /usr/local/bin/s3-sftp-proxy
ENTRYPOINT ["./entrypoint.sh"]
 
  • s3-sftp-proxy demo 配置文件
    s3-sftp-proxy.example.toml 文件
host_key_file = "./host_key"
[buckets.test]
endpoint = "http://minio:9000"
s3_force_path_style = true
disable_ssl = true
bucket = "demoapp"
region = "us-east-1"
max_object_size = 65536
writable = true
readable = true
listable = true
auth = "test"
keyboard_interactive_auth = false
​
[buckets.test.credentials]
aws_access_key_id = "dalongdemo"
aws_secret_access_key = "dalongdemo"
​
[auth.test]
type = "inplace"
​
[auth.test.users.user01]
password = "test"
[auth.test.users.user02]
password = "test"
 
 

启动&&测试

  • 启动
docker-compose up -d
  • 效果

使用s3-sftp-proxy 暴露minio s3 数据为sftp 访问-LMLPHP

  • sftp 连接
sftp -P 10022 user01@localhost

使用s3-sftp-proxy 暴露minio s3 数据为sftp 访问-LMLPHP

使用goreleaser 构建以及发布

  • goreleaser 说明
    goreleaser 是一个很不错的golang 软件构建 工具，可以快速的构建支持跨平台的二进制软件包，同时还支持rpm，deb 包的制作
  • 初始化goreleaser 项目
    goreleaser 提供了方便的cli 命令可以快速生成构建配置文件
 
goreleaser init
  • 修改文件,添加多平台支持
    .goreleaser.yml
 
.# This is an example goreleaser.yaml file with some sane defaults.
# Make sure to check the documentation at http://goreleaser.com
project_name: s3-sftp-proxy
before:
  hooks:
    # you may remove this if you don't use vgo
    - sh goproxy.sh
    - go mod download
builds:
- env:
  - CGO_ENABLED=0
  goos:
  - freebsd
  - windows
  - darwin
  goarch:
  - amd64
  - arm
  - arm64
archive:
  replacements:
    darwin: Darwin
    linux: Linux
    windows: Windows
    386: i386
    amd64: x86_64
checksum:
  name_template: 'checksums.txt'
changelog:
  sort: asc
  filters:
    exclude:
    - '^docs:'
    - '^test:'
 

说明
添加了一个hooks，方便支持goproxy 的加速构建

  • 本地构建
    注意goreleaser 是基于tag 的构建管理，你需要包含tag 分支可以使用git tag 进行操作
goreleaser release --skip-publish --rm-dist

效果
使用s3-sftp-proxy 暴露minio s3 数据为sftp 访问-LMLPHP

  • github release 支持
    goreleaser 可以快速的发布构建好的二进制文件到github，但是提前是需要 GITHUB_TOKEN ,可以直接在github 创建
    添加 GITHUB_TOKEN 环境变量
export GITHUB_TOKEN=xxxxxxxxxxxx

发布

goreleaser release --rm-dist

效果

   • releasing using goreleaser 0.101.0...
   • loading config file file=.goreleaser.yml
   • RUNNING BEFORE HOOKS
      • running sh goproxy.sh
      • running go mod download
   • GETTING AND VALIDATING GIT STATE
      • releasing v1.9, commit e7fa48b35d11406c2a7d8945cfa450865264aba3
   • PARSING TAG      
   • SETTING DEFAULTS 
      • LOADING ENVIRONMENT VARIABLES
      • SNAPSHOTING      
      • GITHUB RELEASES  
      • PROJECT NAME     
      • ARCHIVES         
      • BUILDING BINARIES
      • LINUX PACKAGES WITH NFPM
      • SNAPCRAFT PACKAGES
      • CALCULATING CHECKSUMS
      • SIGNING ARTIFACTS
      • DOCKER IMAGES    
      • ARTIFACTORY      
      • S3               
      • HOMEBREW TAP FORMULA
      • SCOOP MANIFEST   
   • SNAPSHOTING      
      • pipe skipped error=not a snapshot
   • CHECKING ./DIST  
      • --rm-dist is set, cleaning it up
   • WRITING EFFECTIVE CONFIG FILE
      • writing config=dist/config.yaml
   • GENERATING CHANGELOG
      • writing changelog=dist/CHANGELOG.md
   • LOADING ENVIRONMENT VARIABLES
   • BUILDING BINARIES
      • building binary=dist/darwin_amd64/s3-sftp-proxy
      • building binary=dist/windows_amd64/s3-sftp-proxy.exe
      • building binary=dist/freebsd_amd64/s3-sftp-proxy
      • building binary=dist/freebsd_arm_6/s3-sftp-proxy
   • ARCHIVES         
      • creating archive=dist/s3-sftp-proxy_1.9_Darwin_x86_64.tar.gz
      • creating archive=dist/s3-sftp-proxy_1.9_Windows_x86_64.tar.gz
      • creating archive=dist/s3-sftp-proxy_1.9_freebsd_x86_64.tar.gz
      • creating archive=dist/s3-sftp-proxy_1.9_freebsd_armv6.tar.gz
   • LINUX PACKAGES WITH NFPM
   • SNAPCRAFT PACKAGES
      • pipe skipped error=no summary nor description were provided
   • CALCULATING CHECKSUMS
      • checksumming file=s3-sftp-proxy_1.9_Darwin_x86_64.tar.gz
      • checksumming file=s3-sftp-proxy_1.9_freebsd_x86_64.tar.gz
      • checksumming file=s3-sftp-proxy_1.9_freebsd_armv6.tar.gz
      • checksumming file=s3-sftp-proxy_1.9_Windows_x86_64.tar.gz
   • SIGNING ARTIFACTS
      • pipe skipped error=artifact signing is disabled
   • DOCKER IMAGES    
      • pipe skipped error=docker section is not configured
   • PUBLISHING       
      • S3               
         • pipe skipped error=s3 section is not configured
      • HTTP PUT         
         • pipe skipped error=put section is not configured
      • ARTIFACTORY      
         • pipe skipped error=artifactory section is not configured
      • DOCKER IMAGES    
      • SNAPCRAFT PACKAGES
      • GITHUB RELEASES  
         • creating or updating release repo=rongfengliang/s3-sftp-proxy-docker tag=v1.9
         • release updated url=https://github.com/rongfengliang/s3-sftp-proxy-docker/releases/tag/v1.9
         • uploading to release file=dist/checksums.txt name=checksums.txt
         • uploading to release file=dist/s3-sftp-proxy_1.9_Darwin_x86_64.tar.gz name=s3-sftp-proxy_1.9_Darwin_x86_64.tar.gz
         • uploading to release file=dist/s3-sftp-proxy_1.9_freebsd_x86_64.tar.gz name=s3-sftp-proxy_1.9_freebsd_x86_64.tar.gz
         • uploading to release file=dist/s3-sftp-proxy_1.9_Windows_x86_64.tar.gz name=s3-sftp-proxy_1.9_Windows_x86_64.tar.gz
         • uploading to release file=dist/s3-sftp-proxy_1.9_freebsd_armv6.tar.gz name=s3-sftp-proxy_1.9_freebsd_armv6.tar.gz
      • HOMEBREW TAP FORMULA
         • pipe skipped error=brew section is not configured
      • SCOOP MANIFEST   
         • pipe skipped error=scoop section is not configured
   • release succeeded after 26.53s

参考资料

https://github.com/moriyoshi/s3-sftp-proxy
https://github.com/rongfengliang/s3-sftp-proxy-docker
https://goreleaser.com/
https://github.com/goreleaser/goreleaser

05-28 17:51
Powered by LMLPHP ©2024 S3 0.053857