Skip to main content

linux - Mountable filesystem that automatically splits files


Is there a [virtual-]filesystem that can automatically split files in storage but present them to the user as single files?


For example:


The user sees: /user/files/10TBfile
But it is stored as: /backingstorage/user/files/10TBfile.{1..100}

Basically the same way a split archive works but in real time. I feel like it should be possible since it's basically storing each virtual disk block as a separate file but I don't know of any existing solutions for it.


If you're curious, the end goal might be something similar to this question: Divide out local files in different servers with limited space with rsync except that I have single large files that need to be split and the files need to be updated in real-time so a daily cron/rsync and split tar are out of the question. I already have the remote drives mounted so I just need a way to split the file and present it as a single file to the user.


Thanks!



Answer



What you want is chunkfs:



ChunkFS is a FUSE based filesystem that allows you to mount an arbitrary file or block device as a directory tree of files that each represent a chunk of user-specified size of the mounted file.



It was written for the same purpose as yours:



ChunkFS was originally written for making space-efficient incremental backups of encrypted filesystem images using rsync. Using the --link-dest option of rsync, you can create incremental backups from the ChunkFS-mounted image where any chunk that hasn't changed since the last backup will be a hard link to the corresponding chunk from the previous backup.



Comments

Post a Comment

Popular Posts

keyboard - Is there any utility/method to change Windows key bindings to type rare chars to currently empty bindings?

I'm currently typing this post with my windows XP machine and (Spanish) keyboard, and I'd like to add some extra symbols to my text. I could open the "char map" windows utility, look for the desired symbols, and paste them. But I'd like something quickier. For example, when I'm using my OSX Mac at work, I can easily add a ©, ™, ® or similar symbols, just pressing some weird ALT-GR + G / H / J, key combinations. In my (Spanish) keyboard mapping, these combinations are empty, as they don't produce any char at all, which, on the other hand, is perfectly normal and desirable. So, I thought: Why couldn't I add some extra key mappings on top of my currently empty ALT-GR + G/J/H Keys in my Spanish keyboard, and thus, being able to quickly type these special symbols? So that's my question: Is there any utility/method to achieve that effect under windows? (My version is XP). I've even googled this for some time but no luck. I've been a long term Hot...
Post a Comment
Read more

virtualization - How to select paravirtualization interface in VirtualBox?

Given a windows 8 host system (Intel Core i5) and a Linux Fedora host, I would like to determine the optimal setting for the paravirtual interface. Options are none Default Legacy minimal Hyper-V KVM This page suggest the selection is only based on the guest system: The biggest change in VirtualBox 5.0 is the introduction of paravirtualization support, bringing higher performance and time-keeping accuracy to supported guest operating systems (Hyper-V on Windows and KVM on Linux). Is that correct? Answer The VirtualBox Manual , in the section titled Paravirtualization providers explains very clearly when each should be used (emphasis added): Minimal: Announces the presence of a virtualized environment. Additionally, reports the TSC and APIC frequency to the guest operating system. This provider is mandatory for running any Mac OS X guests. KVM: Presents a Linux KVM hypervisor interface which is recognized by Linux kernels starting with version 2.6.25. VirtualBox's implementati...
Post a Comment
Read more

How do I transmit a single hexadecimal value serial data in PuTTY using an Alt code?

I am trying to sent a specific hexadecimal value across a serial COM port using PuTTY. Specifically, I want to send the hex codes 9C, B6, FC, and 8B. I have looked up the Alt codes for these and they are 156, 182, 252, and 139 respectively. However, whenever I input the Alt codes, a preceding hex value of C2 is sent before 9C, B6, and 8B so the values that are sent are C2 9C, C2 B6, and C2 8B. The value for FC is changed to C3 FC. Why are these values being placed before the hex value and why is FC being changed altogether? To me, it seems like there is a problem internally converting the Alt code to hex. Is there a way to directly input hex values without using Alt codes in PuTTY? Answer What you're seeing is just ordinary text character set conversion. As far as PuTTY is concerned, you are typing (and reading) text , not raw binary data, therefore it has to convert the text to bytes in whatever configured character set before sending it over the wire. In other words, when y...
Post a Comment
Read more

Desktop reboots itself on sleep or hibernate

I have been using an ASUS M2NPV-VM motherboard for main home desktop workstation, operating Windows Vista x64. This computer has right from day one not been able to enter hibernate or standby; after Windows performs its final actions and brings the machine down, it would automatically revive itself for a reboot. Updating to the second latest BIOS (1201)has not helped (the latest BIOS revision would induce video refresh problems rendering it unusable). I have been reading related discussions on incidents similar to mine to no avail of a true workable solution. They appear to be more speculative guesses rather than actual knowledge on the inner workings of motherboard hardware. Does anybody have any electronic engineering experience on PC energy-saving standards to provide a more informed opinion how to go about getting this to work? More stories: this motherboard could not even reboot properly the first thing i used it. It was due to refresh rate of the onboard GPU, which had no influe...
Post a Comment
Read more