How to Mount Azure Blob Storage as a File System with BlobFuse on Linux 8

How to Mount Azure Blob Storage as a File System with BlobFuse on Linux 8

Azure Blob Storage is a highly scalable and cost-effective object storage solution. While it's excellent for storing large amounts of unstructured data, interacting with it directly via APIs can sometimes be cumbersome for traditional applications or backup tools. This is where BlobFuse comes in.

BlobFuse is an open-source virtual file system driver developed by Microsoft that allows you to mount an Azure Blob Storage container as a file system on your Linux machine. This means you can interact with your blobs using standard Linux file system commands (ls, cp, mv, mkdir, etc.), making integration with existing applications (like Oracle RMAN for backups) seamless.

This blog post will guide you step-by-step on how to install and configure BlobFuse on a Linux 8 (RHEL/CentOS 8 equivalent) server.

Why Use BlobFuse?

• Familiar Interface: Access Azure Blob Storage using standard Linux file system commands.
• Application Compatibility: Enables legacy applications or tools (like RMAN) that expect a file system to interact with cloud storage.
• Performance: Offers good performance for streaming workloads, especially with proper caching configuration.

Prerequisites

Before you begin, ensure you have:

1. A Linux 8 Server: This guide is specifically for RHEL/CentOS 8 or compatible distributions.
2. sudo Access: You'll need root privileges to install packages and modify system files.
3. An Azure Storage Account:

• Storage Account Name: The name of your Azure Storage Account.
• Storage Account Key: One of the access keys (key1 or key2) from your Azure Storage Account. Keep this secure!
• Blob Container Name: The name of the specific container within your storage account that you wish to mount. Ensure this container already exists.

Step-by-Step Guide

Step 1: Add the Microsoft Azure Linux Software Repository

BlobFuse is distributed through Microsoft's official Linux package repositories. You need to add this repository to your system first.

sudo rpm -Uvh https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm

This command downloads and installs the RPM package that configures the Microsoft repository on your system.

Linux 7 :

sudo rpm -Uvh https://packages.microsoft.com/config/rhel/7/packages-microsoft-prod.rpm

Step 2: Install BlobFuse

Once the repository is added, you can install BlobFuse using your system's package manager.

sudo yum install blobfuse -y

The -y flag automatically confirms any prompts during the installation process.

Step 3: Configure BlobFuse Credentials

BlobFuse needs to know which Azure Storage Account and container to connect to, along with the authentication credentials. This is typically done via a configuration file.

Create a directory for the configuration file:

sudo mkdir -p /etc/blobfuse

Create the BlobFuse configuration file: Open a new file, for example, /etc/blobfuse/config.cfg, using a text editor like vi or nano.

sudo vi /etc/blobfuse/config.cfg

Add the following content to the file. Replace the placeholder values with your actual Azure Storage Account Name, Account Key, and Container Name.

# /etc/blobfuse/config.cfg accountName <YOUR_AZURE_STORAGE_ACCOUNT_NAME> accountKey <YOUR_AZURE_STORAGE_ACCOUNT_KEY> containerName <YOUR_AZURE_BLOB_CONTAINER_NAME> # Optional: Enable block size based caching. This is crucial for performance, # especially with large files like database backups. The value is in MB. # block-size-mb 256

Important Security Note: Storing the accountKey directly in a file can be a security risk. For enhanced security in production environments, consider using environment variables to pass the credentials to BlobFuse. If you opt for environment variables, you would remove the accountName and accountKey lines from config.cfg and set them in your shell session or script before running blobfuse: export AZURE_STORAGE_ACCOUNT="<account_name>" export AZURE_STORAGE_ACCESS_KEY="<account_key>"

1. Set Secure Permissions for the Configuration File: It is critical to restrict access to this file, as it contains sensitive credentials. BlobFuse will refuse to start if the permissions are too open.

   sudo chmod 600 /etc/blobfuse/config.cfg
   
   

   This command sets permissions so that only the root user (owner) can read and write to the file. Verify with ls -l /etc/blobfuse/config.cfg. The output should show -rw-------.

Step 4: Create a Local Mount Point and Temporary Path

BlobFuse requires two local directories:

• A mount point: This is where your Azure Blob Storage container will appear in your Linux file system.
• A temporary path: BlobFuse uses this for caching and buffering data during operations. This should ideally be on a fast local disk (e.g., SSD) with ample free space.

Create the mount point directory:

sudo mkdir /mnt/azure_blob_backups # Choose a descriptive name

Create the temporary path for caching:

sudo mkdir /tmp/blobfuse_cache 
sudo chown <your_user>:<your_group> /tmp/blobfuse_cache # Change ownership to the user who will access the mount 
sudo chmod 777 /tmp/blobfuse_cache # Adjust permissions as per your security policy. 777 is permissive.

Crucial: Ensure the /tmp/blobfuse_cache directory (or your chosen path) has sufficient free space. If it fills up, BlobFuse operations will fail.

Step 5: Mount Azure Blob Storage

Now, you can execute the blobfuse command to mount your Azure Blob Storage container.

sudo blobfuse /mnt/azure_blob_backups --tmp-path=/tmp/blobfuse_cache --config-file=/etc/blobfuse/config.cfg -o allow_other

• /mnt/azure_blob_backups: Your chosen local mount point.
• --tmp-path=/tmp/blobfuse_cache: The temporary directory for caching.
• --config-file=/etc/blobfuse/config.cfg: Path to your BlobFuse configuration file.
• -o allow_other: This FUSE option is essential. It allows users other than the one who initiated the mount (which is root when using sudo) to access the mounted file system. Without this, only root would be able to read/write to /mnt/azure_blob_backups.

Step 6: Verify the Mount

Confirm that the Azure Blob Storage container has been successfully mounted.

Check disk usage:

df -h /mnt/azure_blob_backups

You should see an entry for blobfuse mounted at your specified path.

List directory contents:

ls -l /mnt/azure_blob_backups

This command should display the contents (files and folders) of your Azure Blob Storage container.

Step 7: Unmount Azure Blob Storage (When Needed)

To unmount the BlobFuse file system, use the umount command:

sudo umount /mnt/azure_blob_backups

Step 8: Automount on Boot (Recommended for Production)

For production environments, you'll want your Azure Blob Storage to be automatically mounted every time your server reboots. This is achieved by adding an entry to the /etc/fstab file.

Open /etc/fstab for editing:

sudo vi /etc/fstab


Add the following line to the end of the file:

/etc/blobfuse/config.cfg /mnt/azure_blob_backups fuse.blobfuse _netdev,--tmp-path=/tmp/blobfuse_cache,-oallow_other 0 0

• The first field (/etc/blobfuse/config.cfg) is the configuration file.
• The second field (/mnt/azure_blob_backups) is your mount point.
• fuse.blobfuse: Specifies the file system type.
• _netdev: This crucial option ensures that the mount operation waits for network interfaces to be online before attempting to mount BlobFuse.
• --tmp-path=/tmp/blobfuse_cache,-oallow_other: These are the options passed to BlobFuse, separated by commas.
• 0 0: Specifies no dump and no file system check (fsck) at boot.

Test the fstab entry without rebooting: It's always a good practice to test your fstab entry before rebooting to avoid boot issues.

sudo mount -a

If there are no errors, your fstab entry is likely correct. Verify the mount again with df -h /mnt/azure_blob_backups.

Important Considerations

Security:

• Account Keys: Treat your Azure Storage Account Keys like passwords. If possible, restrict access to the config.cfg file to only the necessary user (e.g., root).
• Shared Access Signatures (SAS): For more granular control and reduced risk, consider using a Shared Access Signature (SAS) token instead of the full account key. You can generate a SAS token with specific permissions (e.g., read, write, list) and an expiry time. You would then use sasToken <your_sas_token> in your config.cfg instead of accountKey.

Performance and Caching:

• The block-size-mb parameter in config.cfg can significantly impact performance. Adjust it based on the typical size of files you'll be transferring. For large database backups, a larger block size (e.g., 256 MB or 512 MB) can be beneficial.
• Ensure your --tmp-path is on a fast, local disk with plenty of free space. This is where BlobFuse caches data.

Error Handling: When integrating with backup scripts (like RMAN), ensure your scripts include error handling for situations where the BlobFuse mount might be unavailable or operations fail.

Please do like and subscribe to my youtube channel: https://www.youtube.com/@foalabs If you like this post please follow,share and comment