AdSense

網頁

2024/5/6

GCP Cloud Storage bucket mount to Linux

將GCP Cloud Storage bucket掛載為Linux檔案系統磁碟的方式如下。


本範例的Linux為GCP VM instance，作業系統為Debian GNU/Linux, 12 (bookworm)。

建立Service Account Key

建立可存取Cloud Storage bucket權限的service account和key。

在GCP console上方搜尋欄位輸入[Service Accounts]，點選[Service Accounts]。



在[Service accounts]頁面上方點選[＋ CREATE SERVICE ACCOUNT]。



在[Create service account]頁面，
第一步輸入service account名稱；
第二步在[Role]欄位找到並選擇[Storage Object Admin]；
第三部跳過直接按[DONE]建立。



進入建立的service account頁面，在[KEYS]頁籤下找到並點選[ADD KEY]，選擇[Create a new key]。



在[Create private key]對話框選擇[JSON]，點選[CREATE]便會自動下載key為JSON檔到本機。



Service account key的JSON內容如下：

project-id-1-243f9679489b.json

{
  "type": "service_account",
  "project_id": "project-id-1",
  "private_key_id": "243f9679489bd2c9d49263f90ba04af689ead799",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDWxn1QiE3v3Ufd\n84bTHRbjWY0ggPv8IiAGkyPP7kr1M/DC74Zg965aXUKig7YIAiyaACTdAVxcwxhe\ndq3aVY5HyIjocIOiQZaRrdZdzoRtUckgSsLd77KXTvxplrca2yzdlh+4qZP3j4go\nvTvUoUIaAa1VPJD15zzRg0Ysyn4AgpxZLoXnvWkmdk0nkXbXZwBjSV+BhoZh5u39\nfkNQEiuiyL/lHlTgzEeFve+qN8vDUjNSK+nXBXf2Bf+eO6nslOYx0Z4EieJVaC61\nTC8/IWvFMcPr123ofnjc2qGkG+Xg447EPL0ByELJVVzkhh6fblJU2T6RSXGsYl4Y\nMQ01ofQbAgMBAAECggEAIbvoH50OeMK4hgZjwocK0EeBi33BlnbvBs6s96eFrdaB\nq2yfHl3ozG1iM6UdE0xGyyLYp2njovegZ+ibd/QA4V+AZC8/+F1K6V4fkc5dLc/Z\nwluxGIDh+91pIgCNNAesQrBQvTAi0JCCd2ZO88p8V8S0VUiqZXqeKoju8xwUXGnd\njF1UVvlM8DyuwNvVrRumz58stG8Bi+CoY46tBIOweoEwJ8FloGFzdBGRNOp5Cnb1\nx/WjPST156mnrN3A7pNWDiuFLjGATtjIWytQmiAPBNW4GnNaUMlzkTKUOh/KMPUt\nv5PAnDhSx0DdqBBG0CeBudkfMTxFowsJhqIviHNlQQKBgQD72fCdyoCT4IdPNJXs\nwQ78wVzKz2Mg3Kl1aviCCKVVTanWVGZgafNSX5L9U6j/131Bko0308SqDvBXnOWf\neClFXcxlzu8N7zYPjAsrNfUtmiIhycdjtYW5akLuOspyRSRn687qeyfjHsAXckvm\nAbEbsMYqxufPLCNLuT030vd6+wKBgQDaUDMjH1PMW0Gb9z7bR6TWrY4iY1d/vojk\n39c0lit2nvVyTvMPhQmwj+37950ICGHVSHV9XFsuduAWP2zLFkP4ZfUz/Rm+YYJ+\n0+BWJP5rykQ+FAQ7iXuA81sTGiybWzvfQpGMEAIUDnfvg9Y5K05MOA2Dh4ML0w3K\ncp+N7WYhYQKBgQCjeABPyuLOGQx6gp1VgLHjqPY7xYSIDfIH/piyfj4iMjisnzec\nGOkQbGyR9T1aYlowt5jlxZR+9RdXJsV8TpgrKAjFhB1yI/Qo/zM4ptsu8D57Q+ch\ndu8HPEuZKZvBs9M/iLVTSQDI0GrKTSuqbSqx2HmnxlcsB3AZdHChzVNJBQKBgH/t\nDPogylbuhpShHhkac1JE4wvtkIFPoiITtKSg9pkM7qDr+Gv4tkPbI0ZJw5/IYaO8\nUPSDoP6qNAd1ZQfgZTXycOvk/SfdNb2h5Ew2Y0vAe5qw2gYVOuTGNXENHmRu4reh\n523el4kvLUKRy382jCwHkIupZbYRF3yxDCP7n6QhAoGAbiG6E1MC4fUlWtHiwhbA\nZriagqjDX6ck+s1qrJh+g/D/8kcdOebhCoFO7QGXP8E0lLS69C2v4Tet6mgKlmNZ\nTbuG08qiqTOGH7DyirscJCTIJyxEufJ6ARLP+V+B+mWpUHISMpuS10AAkfq9s4IA\n7Hf0tVrenauABeab3ybHH7U=\n-----END PRIVATE KEY-----\n",
  "client_email": "mount-bucket@project-id-1.iam.gserviceaccount.com",
  "client_id": "116721481271352847618",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/mount-bucket%40project-id-1.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}


建立Cloud Storage bucket

前往[Cloud Storage]建立一個bucket，建立時使用以下設定：

  • [Location type] - 選擇[Multi-region]。
  • [Prevent public access] - 勾選[Enforce public access prevention on this bucket]。
  • [Access control] - 選擇[Uniform]。

範例的bucket名稱為bucket-202405061125。


安裝Cloud Storage FUSE

參考官方教學「Mount a Cloud Storage bucket using Cloud Storage FUSE - Install Cloud Storage FUSE」部分。


掛載Cloud Storage bucket

輸入sudo mkdir [MOUNT_DIRECTORY]建立待會要掛載bucket的目錄。例如sudo mkdir "$HOME/mnt"建立$HOME/mnt目錄。

$ mkdir "$HOME/mnt"

輸入gcsfuse --key-file [KEY_FILE_PATH] [BUCKET_NAME] [MOUNT_DIRECTORY]將bucket掛載為檔案系統目錄。[KEY_FILE_PATH]為service account key檔案的檔案路徑。

$ gcsfuse --key-file project-id-1-243f9679489b.json bucket-202405061125 "$HOME/mnt"
{"timestamp":{"seconds":1714988252,"nanos":717988969},"severity":"INFO","message":"Value of [key-file] resolved from [project-id-1-243f9679489b.json] to [/home/john/project-id-1-243f9679489b.json]\n"}
{"timestamp":{"seconds":1714988252,"nanos":718555904},"severity":"INFO","message":"Start gcsfuse/2.0.1 (Go version go1.22.1) for app \"\" using mount point: /home/john/mnt\n"}
{"timestamp":{"seconds":1714988252,"nanos":718817577},"severity":"INFO","message":"GCSFuse mount command flags: {\"AppName\":\"\",\"Foreground\":false,\"ConfigFile\":\"\",\"MountOptions\":{},\"DirMode\":493,\"FileMode\":420,\"Uid\":-1,\"Gid\":-1,\"ImplicitDirs\":false,\"OnlyDir\":\"\",\"RenameDirLimit\":0,\"CustomEndpoint\":null,\"BillingProject\":\"\",\"KeyFile\":\"/home/john/project-id-1-243f9679489b.json\",\"TokenUrl\":\"\",\"ReuseTokenFromUrl\":true,\"EgressBandwidthLimitBytesPerSecond\":-1,\"OpRateLimitHz\":-1,\"SequentialReadSizeMb\":200,\"MaxRetrySleep\":30000000000,\"StatCacheCapacity\":20460,\"StatCacheTTL\":60000000000,\"TypeCacheTTL\":60000000000,\"HttpClientTimeout\":0,\"MaxRetryDuration\":-1000000000,\"RetryMultiplier\":2,\"LocalFileCache\":false,\"TempDir\":\"\",\"ClientProtocol\":\"http1\",\"MaxConnsPerHost\":100,\"MaxIdleConnsPerHost\":100,\"EnableNonexistentTypeCache\":false,\"StackdriverExportInterval\":0,\"OtelCollectorAddress\":\"\",\"LogFile\":\"\",\"LogFormat\":\"json\",\"ExperimentalEnableJsonRead\":false,\"DebugFuseErrors\":true,\"DebugFuse\":false,\"DebugFS\":false,\"DebugGCS\":false,\"DebugHTTP\":false,\"DebugInvariants\":false,\"DebugMutex\":false}"}
{"timestamp":{"seconds":1714988252,"nanos":718941062},"severity":"INFO","message":"GCSFuse mount config flags: {\"CreateEmptyFile\":false,\"Severity\":\"INFO\",\"Format\":\"json\",\"FilePath\":\"\",\"LogRotateConfig\":{\"MaxFileSizeMB\":512,\"BackupFileCount\":10,\"Compress\":true},\"MaxSizeMB\":-1,\"CacheFileForRangeRead\":false,\"CacheDir\":\"\",\"TtlInSeconds\":-9223372036854775808,\"TypeCacheMaxSizeMB\":4,\"StatCacheMaxSizeMB\":-9223372036854775808,\"EnableEmptyManagedFolders\":false,\"ConnPoolSize\":1}"}
{"timestamp":{"seconds":1714988252,"nanos":868867811},"severity":"INFO","message":"File system has been successfully mounted."}

輸入mount | grep [MOUNT_DIRECTORY]檢視bucket是否已掛載到目錄。例如範例為mount | grep bucket-202405061125。

$ mount | grep bucket-202405061125
bucket-202405061125 on /home/john/mnt type fuse.gcsfuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1001,default_permissions)


卸載

輸入fusermount -u [MOUNT_DIRECTORY]即可將掛載的目錄卸載。

$ fusermount -u "$HOME/mnt"


標籤： ,

沒有留言:

張貼留言

訂閱： 張貼留言 (Atom)

AdSense