Upgrading to Blobfuse2 from Blobfuse1 Step by Step

Why Blobfuse2?

Blobfuse2 is not merely an update; it's a re-engineered iteration of its predecessor, bringing with it a suite of pivotal enhancements. Expect improved performance, meticulously optimized for achieving higher throughput and significantly lower latency, especially critical for large file operations inherent in database backups. You will also discover enhanced stability, characterized by superior handling of concurrent operations and a more robust approach to edge cases. Blobfuse2 empowers you with richer features, granting more granular control over caching mechanisms, logging, and diverse authentication methods. Furthermore, it proudly boasts better compatibility; while the fundamental nature of FUSE still imposes inherent limitations for highly specialized I/O (as seen in certain RDBMS operations), Blobfuse2 introduces options that can profoundly mitigate these challenges. For Oracle RMAN, the ability to precisely fine-tune caching behavior and ensure immediate, consistent writes is absolutely crucial, a capability that Blobfuse2 facilitates with remarkable effectiveness.

Essential Prerequisites Before Embarking on Your Upgrade

 Root/Sudo Access is an absolute necessity for executing all installation and mounting commands. Be acutely aware of the Temporary Downtime involved; the existing Blobfuse v1 mount point will be unmounted, rendering your backup destination temporarily unavailable. Therefore, meticulous planning for a suitable maintenance window is advised. This guide operates under the assumption of an Existing Blobfuse v1 Mount located at /backup_DB, with its associated temporary cache residing at /u01/blobcache/.

You will require your precise Azure Storage Account Credentials: this includes your Storage Account Name (represented as <YOUR_STORAGE_ACCOUNT_NAME>), your Account Key (denoted as <YOUR_STORAGE_ACCOUNT_KEY>), your Container Name (specified as <YOUR_CONTAINER_NAME>), and the corresponding Endpoint (https://<YOUR_STORAGE_ACCOUNT_NAME>.blob.core.windows.net).

Crucially, identify your Oracle User Information. This encompasses your Oracle OS user (typically oracle) and its primary group (commonly oinstall or dba), as these are indispensable for correctly setting file system permissions. It is paramount to determine the exact UID and GID of your oracle user and oinstall group. For illustrative purposes within this guide, we will use uid=101 and gid=1001; you can readily ascertain your specific values using commands like id -u oracle and grep oinstall /etc/group.

Finally, confirm that your Local Cache Path /u01/blobcache/ exists and possesses ample free space. Ideally, this path should reside on a fast, local disk, such as an ephemeral SSD, to optimize Blobfuse2's temporary caching operations. This designated path must also be writable by the user responsible for mounting Blobfuse2.

Step 1: Unmount and Clean Blobfuse v1

This initial step is critical. It ensures that no lingering processes are holding onto the old mount point and that any stale cached data from the previous Blobfuse version is thoroughly cleared.

Begin by unmounting the existing Blobfuse v1 mount point. As root or with sudo, execute the following command:

sudo umount /backup_DB

Should umount report "device is busy," it signifies that other processes are still actively using the mount point. To troubleshoot this, you can identify the culprit processes by running sudo lsof /backup_DB. Exercise extreme caution if you choose to terminate these processes, especially within a production environment. As a less ideal, but sometimes necessary, last resort, a lazy unmount can be attempted: sudo umount -l /backup_DB.

Following the unmount, clean the Blobfuse v1 temporary cache directory. As root or with sudo, run:

sudo rm -rf /u01/blobcache/*

This command will comprehensively remove all cached data associated with the old Blobfuse v1.

Step 2: Uninstall Blobfuse v1

Removing the outdated Blobfuse v1 package is an essential measure to prevent any potential conflicts with the new Blobfuse2 installation.

Execute the appropriate uninstall command based on your specific Linux distribution. For Red Hat, CentOS, or Oracle Linux (RHEL-based systems), utilize:

sudo yum remove blobfuse

For Ubuntu or Debian (Debian-based systems), employ:

sudo apt-get remove blobfuse

Step 3: Install Blobfuse2

Blobfuse2 inherently relies on libfuse3. The installation process, when executed correctly, should typically handle this dependency automatically.

To begin, configure the Microsoft package repository if it's not already set up or if you previously removed it. For RHEL 8/9, Oracle Linux 8/9, or CentOS 8/9, execute:

sudo rpm -Uvh https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm

Note that if you are operating on RHEL 7 or Oracle Linux 7, you would adjust the URL to rhel/7. 


For Ubuntu 20.04 and newer versions, use:

sudo wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb 
sudo dpkg -i packages-microsoft-prod.deb 
sudo apt-get update

Next, proceed to install Blobfuse2. For RHEL-based systems, run:

sudo yum install blobfuse2

For Debian-based systems, use:

sudo apt-get install blobfuse2

Finally, verify your Blobfuse2 installation by simply running:

blobfuse2 version

A successful installation will be confirmed by the display of the installed Blobfuse2 version number.

Step 4: Prepare Blobfuse2 Configuration File (/etc/blobfuse2/config.yaml)

Blobfuse2 adopts a YAML-formatted configuration file, which provides a more structured and feature-rich approach compared to Blobfuse v1's INI-style format.

First, create the necessary directory for the new configuration file:

sudo mkdir -p /etc/blobfuse2/

Then, create the config.yaml file and populate it with the required content. Open the file using a text editor, for instance, sudo vi /etc/blobfuse2/config.yaml. Paste the following YAML structure into the file. It is absolutely crucial that you replace the placeholder values with your actual Azure Storage Account Name, Access Key, Container Name, and Endpoint.

# /etc/blobfuse2/config.yaml 

logging:
  # This setting directs logs to the system log (e.g., /var/log/syslog or /var/log/messages).
  # It is particularly useful for debugging Blobfuse2 operations.
  type: syslog
  # For initial setup and troubleshooting, set to log_debug for highly verbose output.
  # Once stable, consider changing to log_info or log_warning to reduce log volume in production.
  level: log_debug

components:
  # These are the essential internal components that Blobfuse2 utilizes to interact with the OS and Azure.
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  # FUSE-specific options governing the caching of file attributes and directory entries.
  # Higher values can minimize metadata calls to Azure, potentially enhancing performance.
  attribute-expiration-sec: 240
  entry-expiration-sec: 240
  negative-entry-expiration-sec: 120

file_cache:
  # This specifies the path for the temporary cache directory. It is imperative that this path
  # resides on a local, fast disk (such as an ephemeral SSD) and has ample free space,
  # as Blobfuse2 uses it to buffer open files.
  path: /u01/blobcache/
  # This is CRITICAL for Oracle RMAN compatibility: Setting 'timeout-sec' to 0 effectively disables
  # client-side file caching. This action compels data to be written directly to Azure Blob Storage
  # as much as possible, which aligns more favorably with RMAN's expectations for data integrity
  # and immediate persistence, thereby helping to prevent the notorious ORA-27044 errors.
  timeout-sec: 0
  # This parameter sets the maximum size of the cache in MB. A value of 0 indicates no explicit
  # size limit imposed by Blobfuse2 itself; however, the available disk space at the 'path'
  # specified above will remain the ultimate limiting factor.
  max-size-mb: 0

attr_cache:
  # This defines the timeout duration for attribute caching.
  timeout-sec: 7200

azstorage:
  # These are the specific details for your Azure Storage account.
  type: block # Always use 'block' for standard Azure Block Blob Storage
  account-name: <YOUR_STORAGE_ACCOUNT_NAME>
  account-key: <YOUR_STORAGE_ACCOUNT_KEY>
  container: <YOUR_CONTAINER_NAME>
  # This is the complete endpoint URL for your Azure Blob Storage account.
  endpoint: https://<YOUR_STORAGE_ACCOUNT_NAME>.blob.core.windows.net
  mode: key # The authentication mode: 'key' uses the account key. Other options include 'msi' (Managed Identity) or 'spn' (Service Principal).

After pasting the content, save and close the config.yaml file.

Finally, secure the permissions for the configuration file by executing:

sudo chmod 600 /etc/blobfuse2/config.yaml

This step is absolutely essential to safeguard your Azure Storage account key from any unauthorized access.

Step 5: Prepare Blobfuse2 Temporary Path and Mount Point

This preparatory step ensures that all necessary directories are in place and possess the correct permissions, allowing Blobfuse2 to operate effectively and Oracle RMAN to write successfully.

First, prepare the temporary cache directory (/u01/blobcache/). Verify that this directory exists. It must be writable by the user who will execute the blobfuse2 mount command. This could be root if you're using sudo for mounting, or the oracle user if you intend to mount it directly under the oracle user's context.

# Ensure the directory exists
sudo mkdir -p /u01/blobcache/
# Set ownership and permissions. If mounting with sudo, root needs write.
sudo chown root:root /u01/blobcache/
sudo chmod 755 /u01/blobcache/

Next, prepare the Blobfuse2 mount point (/backup_DB). This specific directory is where your Azure Blob Storage container will manifest as a local file system. 


# Create the mount point directory if it doesn't already exist

sudo mkdir -p /backup_DB

# Set ownership and permissions for the mount point.
# This example assumes 'oracle' as the user and 'oinstall' as the group. Adjust these if your Oracle user/group differs.
sudo chown oracle:oinstall /backup_DB
sudo chmod 775 /backup_DB # This allows the owner (oracle) and group (oinstall) to read, write, and execute.

Step 6: Mount Azure Blob Storage with Blobfuse2

Now, proceed to execute the actual mount operation using Blobfuse2. In this crucial step, we will explicitly include the uid and gid options to ensure that the ownership of files and directories created within the mounted filesystem is correctly assigned to your Oracle user and group.

Mount the container using Blobfuse2. As root or with sudo, use the following command:

# As root or with sudo
# The --disable-writeback-cache=true option is crucial for RMAN compatibility.
# The -o uid and -o gid parameters explicitly set the owner of files/directories on the mount.
sudo blobfuse2 mount /backup_DB \
    --config-file=/etc/blobfuse2/config.yaml \
    -o allow_other \
    -o uid=101 \
    -o gid=1001 \
    --disable-writeback-cache=true \
    --log-level=LOG_DEBUG

Let's break down the significance of these mount command options:

  • -o allow_other: This FUSE option is essential when you are mounting Blobfuse2 as root (typically via sudo). It grants access to the mounted file system to users other than the one who initiated the mount (in this case, the oracle user). Without this, only root would be able to read and write to /backup_DB.

  • -o uid=101: This parameter explicitly sets the User ID (UID) for all files and directories created or accessed within the mounted filesystem to 101, which corresponds to your oracle user's UID.

  • -o gid=1001: Similarly, this parameter explicitly sets the Group ID (GID) for all files and directories within the mounted filesystem to 1001, aligning with your oinstall group's GID.

  • --disable-writeback-cache=true: This is CRITICAL for RMAN. This option compels Blobfuse2 to write data to Azure Blob Storage immediately, effectively bypassing certain write-back caching mechanisms. This behavior is vital for applications like RMAN that demand immediate data persistence and can significantly help in preventing the notorious ORA-27044 errors.

  • --log-level=LOG_DEBUG: Keeping verbose logging enabled during the initial setup is highly recommended. This will direct detailed information to your system logs (e.g., /var/log/syslog or /var/log/messages), providing invaluable insights for debugging any remaining issues.

Step 7: Verify Blobfuse2 Mount and Permissions

After mounting, it's imperative to confirm that the mount is active and, most importantly, fully accessible and writable by the oracle user.

First, verify the mount status by running:

mount | grep blobfuse2
df -h /backup_DB

You should observe /backup_DB clearly listed as a mount point managed by blobfuse2.







Please do like and subscribe to my youtube channel: https://www.youtube.com/@foalabs If you like this post please follow,share and comment