Project

General

Profile

Actions

Task #6899

closed

Task #6869: Challenge OpenStack and OpenNebula with ucloud

Task #6897: Create ucloud-image service

Create ucloud-file-scan service

Added by Nico Schottelius over 5 years ago. Updated 11 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
06/28/2019
Due date:
% Done:

50%

Estimated time:
PM Check date:

Description

Description

  • Has a base directory (like /var/www/
    • to be injected by enviroment
  • Below this directory exists subdirectories PER user
    • Like /var/www/nico

Objective

The ucloud-file-scan service scans the base path every x minutes (we might be able to trigger this by cron job) and creates etcd entries.
Users can list their files by ucloud-cli. Files are created off band with sftp.

How it interacts with the other systems

  • Found files are created in etcd below /v1/files/$UUID
    • Each file gets a random UUID
  • Json contains at least:
    • owner
    • some checksum (sha512sum maybe)
    • date created
    • size (in bytes)

How to extend ucloud-api and ucloud-cli

Support the following command(s)

  • ucloud-cli files-list

Upload and delete is NOT supported by the CLI, but will be using sftp.

Note: we COULD wrap sftp for the user into

  • ucloud-cli files-upload file [file...]

which in turn calls

  • sftp ... files.datacenterlight.ch

(this is earliest for v2)

Improving scanning speed

In theory we can also use inotify and events for rescanning. However if we miss an event, we miss the file/image upload. Thus we need to regularly scan, but we can ALSO have an event based system as well.

(this is earliest for v3)

Actions #1

Updated by Ahmed Bilal over 5 years ago

  • Status changed from New to Seen
Actions #2

Updated by Ahmed Bilal over 5 years ago

  • Status changed from Seen to In Progress
Actions #3

Updated by Ahmed Bilal over 5 years ago

v1 done.

  • We are using extended attributes to mark files that are tracked by ucloud-file-scan
  • JSON Entry for File in Etcd follows format described here
  • Code can be found here
Actions #4

Updated by Ahmed Bilal over 5 years ago

  • % Done changed from 0 to 50
Actions #5

Updated by Ahmed Bilal over 5 years ago

  • Status changed from In Progress to Feedback
Actions #6

Updated by Ahmed Bilal over 5 years ago

ucloud user list --files to list all user files
ucloud user list --vms to list all vms files

Actions #7

Updated by Ahmed Bilal over 5 years ago

  • Status changed from Feedback to In Progress
Actions #8

Updated by Ahmed Bilal almost 4 years ago

  • Assignee deleted (Ahmed Bilal)
Actions #9

Updated by Nico Schottelius 11 months ago

  • Status changed from In Progress to Rejected
Actions

Also available in: Atom PDF