Active/Active NFS on Clustered Filesystems: Basic Design

In the last couple of years, clustering support for Samba has really taken off. It’s now possible to take a clustered filesystem (such as GFS2, OCFS2, or GPFS), and serve that filesystem via samba from multiple nodes. The next step is do the same with NFS. Since NFSv2/3 are rapidly becoming legacy protocols, we want to base this on NFSv4.

Here’s what I envision as a basic overall design…

Start with a cluster of machines with a shared-storage filesystem like GFS2. Each node in the cluster runs knfsd, and its own set of daemons:

  • mountd
  • idmapd
  • rpcsvcgssd
  • …and any other ones that are needed, such as a client id tracking daemon

Multiple IP addresses will float back and forth freely in the cluster. That will be handled by resource management software (rgmanager or something). When a cluster resource group floats to another server, it should just need to move the address. Restarting knfsd should not be allowed (or recommended).

When an IP address moves, any clients that have mounted through that address will need to reclaim their state (locks and opens, eventually we may also want to allow reclaim of delegations too).

One major goal is to ensure that when an IP address floats to another server that we don’t need to put the entire cluster into grace, forcing clients on all of the other addresses to reclaim their state as well.

Advertisements

2 thoughts on “Active/Active NFS on Clustered Filesystems: Basic Design

    1. jtlayton Post author

      Not exactly. You’re probably thinking of pNFS, but that still requires a “master” node to handle metadata support. The idea here is to have multiple nodes that can act more or less independently of one another, mostly just connected by GFS2. This is really the NFSv4 equivalent of Samba/CTDB:

      http://wiki.samba.org/index.php/CTDB_Setup

      It’s possible we can eventually expand the scope of this to allow for pNFS support as well, but it’s really a different thing entirely. That won’t be in the initial implementation.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s