Skip to main content

linux - Understanding metadata usage behavior in a newly converted btrfs filesystem

I have a 5 TB hard drive with 3.3 TiB (as reported by du) of media files in 119k files and directories; the average file size was about 28 MiB. I converted the ext4 partition into btrfs with btrfs-convert. The process took 10.4 hours and was CPU-bound. Right after the conversion:


# btrfs fi df /mnt/btrfs/
Data, single: total=3.03TiB, used=2.21TiB
System, single: total=32.00MiB, used=236.00KiB
Metadata, single: total=1.52TiB, used=1.10TiB

btrfs allocated all space on the hard drive (3.03 + 1.52 TiB ≈ 5 TB), and it apparently put ~1 TiB of file content into metadata chunks (1.10 TiB used). This is unexpected because very few of my files would fit into the leaf nodes.


The standard solution to the huge metadata allocation is to re-balance the metadata. btrfs balance start -m took 8.0 hours and was I/O-bound. Afterwards, it appears that btrfs not only released unused metadata chunks, but also moved file content from metadata chunks into data chunks.


# btrfs fi df /mnt/btrfs/
Data, single: total=3.32TiB, used=3.31TiB
System, single: total=32.00MiB, used=104.00KiB
Metadata, single: total=3.00GiB, used=2.18GiB

Could someone explain what is happening? Why would "metadata" consume 1.10 TiB initially, and why would balancing reduce it to 2.18 GiB? Does btrfs balance move file content from metadata chunks to data chunks?


I used Linux kernel 3.13.0-46. My files have no hard links, xattrs, SELinux contexts, nor ACLs. I have not turned on compression, and have not deleted the ext4 roll back image.

Comments

Post a Comment

Popular Posts

How do I transmit a single hexadecimal value serial data in PuTTY using an Alt code?

I am trying to sent a specific hexadecimal value across a serial COM port using PuTTY. Specifically, I want to send the hex codes 9C, B6, FC, and 8B. I have looked up the Alt codes for these and they are 156, 182, 252, and 139 respectively. However, whenever I input the Alt codes, a preceding hex value of C2 is sent before 9C, B6, and 8B so the values that are sent are C2 9C, C2 B6, and C2 8B. The value for FC is changed to C3 FC. Why are these values being placed before the hex value and why is FC being changed altogether? To me, it seems like there is a problem internally converting the Alt code to hex. Is there a way to directly input hex values without using Alt codes in PuTTY? Answer What you're seeing is just ordinary text character set conversion. As far as PuTTY is concerned, you are typing (and reading) text , not raw binary data, therefore it has to convert the text to bytes in whatever configured character set before sending it over the wire. In other words, when y...
Post a Comment
Read more

linux - Extract/save a mail attachment using bash

Using normal bash tools (ie, built-ins or commonly-available command-line tools), is it possible, and how to extract/save attachments on emails? For example, say I have a nightly report which arrives via email but is a zip archive of several log files. I want to save all those zips into a backup directory. How would I accomplish that? Answer If you're aiming for portability, beware that there are several different versions of mail(1) and mailx(1) . There's a POSIX mailx command, but with very few requirements. And none of the implementations I have seem to parse attachments anyway. You might have the mpack package . Its munpack command saves all parts of a MIME message into separate files, then all you have to do is save the interesting parts and clean up the rest. There's also metamail . An equivalent of munpack is metamail -wy .
Post a Comment
Read more

performance - Single Threaded Qaud Core v.s Hyper-Threading Dual Core

Let's say we have two CPUs, One is Quad Core 3.2 Ghz with 4 Cores, and We have a Dual Core 3.2 Ghz with 2 Cores with 2 threads in each Core (Hyper-Threading). My assumption as a programmer will be, the 4 cores 4 threads should perform faster than 2 cores 4 threads since the second CPU needs to switch between threads in order to emulate 4 cores while the first one doesn't need to perform such switching as each core can perform independently and individually. I want to confirm that my assumption is true, if not please explain why one is better than the other. Answer I do believe thats true - since hyper threading does share some elements - specifically the main execution resources, you'll be able to run 4 full threads at once, rather than waiting for those resources to be freed up. The point of HT is to get better performance with a smaller use of die area - your quad core would generally be a bigger chip - say almost twice as large, than a non HT dual core chip, while a HT...
Post a Comment
Read more

freeze - How do I stop windows 8.1 from freezing when the screen locks

This happens to me on a regular basis if I leave the computer for upwards of 10 minutes. It didnt do so at first but started after a couple of days. This is possibly related to further windows updates although nothing seems to tie in obviously when looking at my update history. I have to hold the power button in to power off. If the screens have switched off aswell they wont come back on, if they haven't I see the login picture and can move the mouse pointer but nothing happens and no combination of keyboard mashes or mouse clicks lets me see the login prompt. In the event log (type event viewer into the start menu) under system before every Critical problem (me powering down the machine without restarting) I get distributedCOM errors talking about this guid: "The server {BF6C1E47-86EC-4194-9CE5-13C15DCB2001} did not register with DCOM within the required timeout." I also get the same error for this 1B1F472E-3221-4826-97DB-2C2324D389AE. This seems to be a common theme and...
Post a Comment
Read more