Project

General

Profile

Task #6899

Task #6869: Challenge OpenStack and OpenNebula with ucloud

Task #6897: Create ucloud-image service

Create ucloud-file-scan service

Added by Nico Schottelius 4 months ago. Updated 3 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
-
Start date:
06/28/2019
Due date:
% Done:

50%

Estimated time:
PM Check date:

Description

Description

  • Has a base directory (like /var/www/
    • to be injected by enviroment
  • Below this directory exists subdirectories PER user
    • Like /var/www/nico

Objective

The ucloud-file-scan service scans the base path every x minutes (we might be able to trigger this by cron job) and creates etcd entries.
Users can list their files by ucloud-cli. Files are created off band with sftp.

How it interacts with the other systems

  • Found files are created in etcd below /v1/files/$UUID
    • Each file gets a random UUID
  • Json contains at least:
    • owner
    • some checksum (sha512sum maybe)
    • date created
    • size (in bytes)

How to extend ucloud-api and ucloud-cli

Support the following command(s)

  • ucloud-cli files-list

Upload and delete is NOT supported by the CLI, but will be using sftp.

Note: we COULD wrap sftp for the user into

  • ucloud-cli files-upload file [file...]

which in turn calls

  • sftp ... files.datacenterlight.ch

(this is earliest for v2)

Improving scanning speed

In theory we can also use inotify and events for rescanning. However if we miss an event, we miss the file/image upload. Thus we need to regularly scan, but we can ALSO have an event based system as well.

(this is earliest for v3)

History

#1

Updated by Ahmed Bilal 4 months ago

  • Status changed from New to Seen
#2

Updated by Ahmed Bilal 4 months ago

  • Status changed from Seen to In Progress
#3

Updated by Ahmed Bilal 4 months ago

v1 done.

  • We are using extended attributes to mark files that are tracked by ucloud-file-scan
  • JSON Entry for File in Etcd follows format described here
  • Code can be found here
#4

Updated by Ahmed Bilal 4 months ago

  • % Done changed from 0 to 50
#5

Updated by Ahmed Bilal 4 months ago

  • Status changed from In Progress to Feedback
#6

Updated by Ahmed Bilal 4 months ago

ucloud user list --files to list all user files
ucloud user list --vms to list all vms files

#7

Updated by Ahmed Bilal 3 months ago

  • Status changed from Feedback to In Progress

Also available in: Atom PDF