forked from jackdoe/sfs
-
Notifications
You must be signed in to change notification settings - Fork 0
therobyouknow/sfs
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
*WARNING: experimental stage, work in progress, DOES NOT USE SSL, DO NOT USE IN PRODUCTION* very basic mysql backed fuse file system. 0: abstract fuse based, mysql backed file system 1. requirements: fuse, mysql, pthreads, c compiler 2. install: download from https://github.com/jackdoe/sfs $EDOTOR common.h #put proper mysql settings into common.h make #DEBUG=1 if you want debug output ./sfs /mnt/fuse 3. info: this is the table that is used: Q("CREATE TABLE IF NOT EXISTS `entries_%s` ( \ `id` bigint(20) NOT NULL AUTO_INCREMENT, \ `path` varchar(255) NOT NULL, \ `data` longblob, \ `obj_type` int(11) NOT NULL, \ `uid` bigint(20) NOT NULL, \ `gid` bigint(20) NOT NULL, \ `mode` bigint(20) NOT NULL, \ `atime` bigint(20) NOT NULL, \ `mtime` bigint(20) NOT NULL, \ `ctime` bigint(20) NOT NULL, \ `btime` bigint(20) NOT NULL, \ PRIMARY KEY (`id`), \ UNIQUE KEY `path` (`path`))",identifier); identifier is a copy of the mysql username (for example root) and the table will be entries_root table is automaticaly created at start, root directory is auto inserted as well mysql user needs to select/update/delete/insert and maybe create if you dont have the table created. 4. idea: i made this as proof of concept to be used in diskless linux based routers, where i can edit their config files remotely - for example /etc or /usr/local/etc, with simple web based application. 5. supported calls: sfs_oper.getattr = sfs_getattr; sfs_oper.readdir = sfs_readdir; sfs_oper.open = sfs_open; sfs_oper.write = sfs_write; sfs_oper.truncate = sfs_truncate; sfs_oper.read = sfs_read; sfs_oper.create = sfs_create; sfs_oper.unlink = sfs_delete; sfs_oper.chmod = sfs_chmod; sfs_oper.chown = sfs_chown; sfs_oper.statfs = sfs_statfs; sfs_oper.mkdir = sfs_mkdir; sfs_oper.readlink = sfs_readlink; sfs_oper.symlink = sfs_symlink; sfs_oper.rename = sfs_rename; sfs_oper.rmdir = sfs_rmdir; sfs_oper.utimens = sfs_utimens; 6. limitations: MAX_PATH_SIZE: 256 chars max data size: mysql's longblob 4gb read/write is extreamly slow on big files currently there is no locking 7. example usage: $ mkdir ~/mysql_backed_fs $ ./sfs ~/mysql_backed_fs/ $ cd ~/mysql_backed_fs mysql_backed_fs $ mkdir test_directory mysql_backed_fs $ cd test_directory/ test_directory $ ls -la total 0 drwxr-xr-x 1 jack staff 0 Dec 27 09:24 . drwxr-xr-x 1 jack staff 0 Dec 24 15:35 .. test_directory $ touch empty_file test_directory $ cat empty_file test_directory $ echo 'not empty anymore' >> empty_file test_directory $ cat empty_file not empty anymore test_directory $ cd .. mysql_backed_fs $ ln -s test_directory/empty_file link mysql_backed_fs $ cat link not empty anymore mysql_backed_fs $ ln -s test_directory link2 mysql_backed_fs $ cat link2/empty_file not empty anymore mysql_backed_fs $ cd .. $ dd if=/dev/random of=blob.dd bs=1m count=1 && md5 blob.dd 1+0 records in 1+0 records out 1048576 bytes transferred in 0.080215 secs (13072073 bytes/sec) MD5 (blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe $ cp blob.dd mysql_backed_fs/ && md5 mysql_backed_fs/blob.dd MD5 (mysql_backed_fs/blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe $ cd mysql_backed_fs as you can see write is very very slow mysql_backed_fs $ dd if=/dev/zero of=./speed.blob.dd bs=1m count=5 5+0 records in 5+0 records out 5242880 bytes transferred in 14.620257 secs (358604 bytes/sec) reading is fine on not very large files tho, because it uses SUBSTRING() and gets only requested offset+size from the mysql mysql_backed_fs $ time cat speed.blob.dd > /dev/null real 0m0.005s user 0m0.001s sys 0m0.004s mysql does not have a way for partial updates on blob fields so there is download/modify (offset+size - usually 65k)/push the whole file and the speed drops like a rock in water :) mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=65k count=1 1+0 records in 1+0 records out 66560 bytes transferred in 0.004575 secs (14548589 bytes/sec) mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=256k count=1 1+0 records in 1+0 records out 262144 bytes transferred in 0.015075 secs (17389082 bytes/sec) mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=512k count=1 1+0 records in 1+0 records out 524288 bytes transferred in 0.046395 secs (11300513 bytes/sec) mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 0.301245 secs (3480808 bytes/sec) mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=2048k count=1 1+0 records in 1+0 records out 2097152 bytes transferred in 1.876782 secs (1117419 bytes/sec) but for small files it is fine
About
fuse based mysql backed file system
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published