glusterfs.git/tests/basic, branch v3.6.1 GlusterFS is a distributed file-system capable of scaling to several petabytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. features/snapview-server: check if the reference to the snapshot world is correct before doing any fop 2014-10-30T18:39:15+00:00 Raghavendra Bhat raghavendra@redhat.com 2014-10-09T12:02:48+00:00 2ebb417e7558a35990b7c2784d25fe38ea975d0e The following operations might lead to problems: * Create a file on the glusterfs mount point * Create a snapshot (say "snap1") * Access the contents of the snapshot * Delete the file from the mount point * Delete the snapshot "snap1" * Create a new snapshot "snap1" Now accessing the new snapshot "snap1" gives problems. Because the inode and dentry created for snap1 would not be deleted upon the deletion of the snapshot (as deletion of snapshot is a gluster cli operation, not a fop). So next time upon creation of a new snap with same name, the previous inode and dentry itself will be used. But the inode context contains old information about the glfs_t instance and the handle in the gfapi world. Directly accessing them without proper check leads to ENOTCONN errors. Thus the glfs_t instance should be checked before accessing. If its wrong, then right instance should be obtained by doing the lookup. Change-Id: I975245b8f6b7fea0a90eb5e36e8149d12457ac10 BUG: 1158791 Reviewed-on: http://review.gluster.org/9007 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com> 
The following operations might lead to problems:
* Create a file on the glusterfs mount point
* Create a snapshot (say "snap1")
* Access the contents of the snapshot
* Delete the file from the mount point
* Delete the snapshot "snap1"
* Create a new snapshot "snap1"

Now accessing the new snapshot "snap1" gives problems. Because the inode and
dentry created for snap1 would not be deleted upon the deletion of the snapshot
(as deletion of snapshot is a gluster cli operation, not a fop). So next time
upon creation of a new snap with same name, the previous inode and dentry itself
will be used. But the inode context contains old information about the glfs_t
instance and the handle in the gfapi world. Directly accessing them without
proper check leads to ENOTCONN errors. Thus the glfs_t instance should be
checked before accessing. If its wrong, then right instance should be obtained
by doing the lookup.

Change-Id: I975245b8f6b7fea0a90eb5e36e8149d12457ac10
BUG: 1158791
Reviewed-on: http://review.gluster.org/9007
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
ec: Fix rebalance issues 2014-10-27T14:18:51+00:00 Xavier Hernandez xhernandez@datalab.es 2014-10-15T07:44:55+00:00 408d4454870e7374bc9c4060c6c23d224d5174a2 Some issues in ec xlator made that rebalance didn't complete successfully and generated some warnings and errors in the log. The most critical error was a race condition that caused false corruption detection when two specific operations were executed sequentially and they shared the same lock. This explains the problem: 1. A setxattr is issued. 2. setxattr: ec locks the inode before updating the xattr. 3. setxattr: The xattr is updated. 4. setxattr: Upper xlator is notified that the operation completed. 5. setxattr: A background task is initiated to update the version of the file. 6. A stat is issued on the same file. 7. stat: Since the lock is already acquired, it's reused. 8. stat: A lookup is issued to determine version and size information of the file. At this point, operations 5 and 8 can interfere. This can make that lookup sees different information on each brick, determining that some bricks are corrupted and incorrectly excluding them from the operation and initiating a self-heal. In some cases this false detection combined with self-heal could lead to invalid updates of the trusted.ec.size xattr, leaving the file smaller than it should be. This only happens if the first operation does not perform a lookup, because chained operations reuse the information returned by the previous one, avoiding this kind of problems. To solve this, now the background update is executed atomically with the posterior unlock. This avoids some reuses of the lock while updating. However this reduces performance because the window in which new requests can reuse the lock is much smaller now. This has been alleviated by using the same technique implemented in AFR (i.e. waiting some time before releasing the lock). Some minor changes also introduced in this patch: * Bug in management of 'trusted.glusterfs.pathinfo' that was writing beyond the allocated space. * Uninitialized variable. * trusted.ec.config was not created for regular files created with mknod. * An invalid state was used in access fop. This is a backport of http://review.gluster.org/8947/ Change-Id: Idfaf69578ed04dbac97a62710326729715b9b395 BUG: 1152903 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8948 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> 
Some issues in ec xlator made that rebalance didn't complete
successfully and generated some warnings and errors in the
log. The most critical error was a race condition that caused
false corruption detection when two specific operations were
executed sequentially and they shared the same lock.

This explains the problem:

1. A setxattr is issued.
2. setxattr: ec locks the inode before updating the xattr.
3. setxattr: The xattr is updated.
4. setxattr: Upper xlator is notified that the operation completed.
5. setxattr: A background task is initiated to update the version
             of the file.
6. A stat is issued on the same file.
7. stat: Since the lock is already acquired, it's reused.
8. stat: A lookup is issued to determine version and size
         information of the file.

At this point, operations 5 and 8 can interfere. This can make that
lookup sees different information on each brick, determining that
some bricks are corrupted and incorrectly excluding them from the
operation and initiating a self-heal. In some cases this false
detection combined with self-heal could lead to invalid updates of
the trusted.ec.size xattr, leaving the file smaller than it should
be.

This only happens if the first operation does not perform a lookup,
because chained operations reuse the information returned by the
previous one, avoiding this kind of problems.

To solve this, now the background update is executed atomically with
the posterior unlock. This avoids some reuses of the lock while
updating. However this reduces performance because the window in
which new requests can reuse the lock is much smaller now. This has
been alleviated by using the same technique implemented in AFR (i.e.
waiting some time before releasing the lock).

Some minor changes also introduced in this patch:

* Bug in management of 'trusted.glusterfs.pathinfo' that was writing
  beyond the allocated space.
* Uninitialized variable.
* trusted.ec.config was not created for regular files created with
  mknod.
* An invalid state was used in access fop.

This is a backport of http://review.gluster.org/8947/

Change-Id: Idfaf69578ed04dbac97a62710326729715b9b395
BUG: 1152903
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/8948
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
ec: Fix self-heal issues 2014-10-22T08:05:19+00:00 Xavier Hernandez xhernandez@datalab.es 2014-10-08T07:20:11+00:00 4522acc20bdd1ca17c053969ef7edce1bb6ede76 Problem: Doing an 'ls' of a directory that has been modified while one of the bricks was down, sometimes returns the old directory contents. Cause: Directories are not marked when they are modified as files are. The ec xlator balances requests amongst available and healthy bricks. Since there is no way to detect that a directory is out of date in one of the bricks, it is used from time to time to return the directory contents. Solution: Basically the solution consists in use versioning information also for directories, however some additional changes have been necessary. Changes: * Use directory versioning: This required to lock full directory instead of a single entry for all requests that add or remove entries from it. This is needed to allow atomic version update. This affects the following fops: create, mkdir, mknod, link, symlink, rename, unlink, rmdir Another side effect is that opendir requires to do a previous lookup to get versioning information and discard out of date bricks for subsequent readdir(p) calls. * Restrict directory self-heal: Till now, when one discrepancy was found in lookup, a self-heal was automatically started. This caused the versioning information of a bad directory to be healed instantly, making the original problem to reapear again. To solve this, when a missing directory is detected in one or more bricks on lookup or opendir fops, only a partial self-heal is performed on it. A partial self-heal basically creates the directory but does not restore any additional information. This avoids that an 'ls' could repair the directory and cause the problem to happen again. With this change, output of 'ls' is always consistent. However, since the directory has been created in the brick, this allows any other operation on it (create new files, for example) to succeed on all bricks and not add additional work to the self-heal process. To force a self-heal of a directory, any other operation must be done on it. For example a getxattr. With these changes, the correct healing procedure that would avoid inconsistent directory browsing consists on a post-order traversal of directoriesi being healed. This way, the directory contents will be healed before healing the directory itslef. * Additional changes to fix self-heal errors - Don't use fop->fd to decide between fd/loc. open, opendir and create have an fd, but the correct data is in loc. - Fix incorrect management of bad bricks per inode/fd. - Fix incorrect selection of fop's target bricks when there are bad bricks involved. - Improved ec_loc_parent() to always return a parent loc as complete as possible. This is a backport of http://review.gluster.org/8916/ Change-Id: Iaf3df174d7857da57d4a87b4a8740a7048b366ad BUG: 1149727 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8946 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> 
Problem: Doing an 'ls' of a directory that has been modified while one
         of the bricks was down, sometimes returns the old directory
         contents.

Cause: Directories are not marked when they are modified as files are.
       The ec xlator balances requests amongst available and healthy
       bricks. Since there is no way to detect that a directory is
       out of date in one of the bricks, it is used from time to time
       to return the directory contents.

Solution: Basically the solution consists in use versioning information
          also for directories, however some additional changes have
          been necessary.

Changes:

 * Use directory versioning:

     This required to lock full directory instead of a single entry for
     all requests that add or remove entries from it. This is needed to
     allow atomic version update. This affects the following fops:

         create, mkdir, mknod, link, symlink, rename, unlink, rmdir

     Another side effect is that opendir requires to do a previous
     lookup to get versioning information and discard out of date
     bricks for subsequent readdir(p) calls.

 * Restrict directory self-heal:

     Till now, when one discrepancy was found in lookup, a self-heal
     was automatically started. This caused the versioning information
     of a bad directory to be healed instantly, making the original
     problem to reapear again.

     To solve this, when a missing directory is detected in one or more
     bricks on lookup or opendir fops, only a partial self-heal is
     performed on it. A partial self-heal basically creates the
     directory but does not restore any additional information.

     This avoids that an 'ls' could repair the directory and cause the
     problem to happen again. With this change, output of 'ls' is
     always consistent. However, since the directory has been created
     in the brick, this allows any other operation on it (create new
     files, for example) to succeed on all bricks and not add additional
     work to the self-heal process.

     To force a self-heal of a directory, any other operation must be
     done on it. For example a getxattr.

     With these changes, the correct healing procedure that would avoid
     inconsistent directory browsing consists on a post-order traversal
     of directoriesi being healed. This way, the directory contents will
     be healed before healing the directory itslef.

 * Additional changes to fix self-heal errors

     - Don't use fop->fd to decide between fd/loc.

         open, opendir and create have an fd, but the correct data is in
         loc.

     - Fix incorrect management of bad bricks per inode/fd.

     - Fix incorrect selection of fop's target bricks when there are bad
       bricks involved.

     - Improved ec_loc_parent() to always return a parent loc as
       complete as possible.

This is a backport of http://review.gluster.org/8916/

Change-Id: Iaf3df174d7857da57d4a87b4a8740a7048b366ad
BUG: 1149727
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/8946
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
test/ec: Fix spurious failures caused by self-heal 2014-10-21T18:39:14+00:00 Xavier Hernandez xhernandez@datalab.es 2014-09-18T16:42:34+00:00 d01b00ae2b124dfdd6905e463533a715f1cedc5b The sha1sum of a file may update the access time of that file. If this happens while a brick is down, as it is forced in the test, that brick doesn't get the update, getting out of sync. When the brick is restarted, self-heal repairs the file, but the test shouldn't access brick contents until self-heal finishes. If this is combined with a kill of another brick before self-heal has finished repairing the file, the volume could become inaccessible. Since the purpose of these tests is only to check ec functionality (there is another test that checks self-heal), the test that corrupts the file has been removed. Additional checks to validate the state of the volume have been added to avoid some timing issues. This is a backport of http://review.gluster.org/8892/ BUG: 1149118 Change-Id: I8a40b7f07fc8ecd2c721bad1bcdd351dd8504155 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8902 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> 
The sha1sum of a file may update the access time of that file.
If this happens while a brick is down, as it is forced in the
test, that brick doesn't get the update, getting out of sync.

When the brick is restarted, self-heal repairs the file, but
the test shouldn't access brick contents until self-heal finishes.
If this is combined with a kill of another brick before self-heal
has finished repairing the file, the volume could become inaccessible.

Since the purpose of these tests is only to check ec functionality
(there is another test that checks self-heal), the test that corrupts
the file has been removed.

Additional checks to validate the state of the volume have been added
to avoid some timing issues.

This is a backport of http://review.gluster.org/8892/

BUG: 1149118
Change-Id: I8a40b7f07fc8ecd2c721bad1bcdd351dd8504155
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/8902
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
ec: Fix incorrect management of healed bricks 2014-10-20T10:40:30+00:00 Xavier Hernandez xhernandez@datalab.es 2014-10-06T14:34:58+00:00 b2a1be0f7bf5e19b266ee57c3287ee9f9170e2f7 The final lookup made to restore final file attributes after a self-heal did clear the mask of bad bricks, causing that the final setattr won't modify any brick at all. This caused that some attriutes, specially the modification time of the file didn't get updated properly. Now the mask of healed bricks is saved before doing the last lookup. It's also used to correctly report the repaired bricks. This is a backport of http://review.gluster.org/8905/ Change-Id: Ib94083c9e1b562515dfb54f9574120f1f031dccc BUG: 1149725 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8906 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
The final lookup made to restore final file attributes after a self-heal
did clear the mask of bad bricks, causing that the final setattr won't
modify any brick at all. This caused that some attriutes, specially the
modification time of the file didn't get updated properly.

Now the mask of healed bricks is saved before doing the last lookup.
It's also used to correctly report the repaired bricks.

This is a backport of http://review.gluster.org/8905/

Change-Id: Ib94083c9e1b562515dfb54f9574120f1f031dccc
BUG: 1149725
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/8906
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
ec: Add state dump support 2014-10-04T15:25:53+00:00 Xavier Hernandez xhernandez@datalab.es 2014-09-30T15:47:26+00:00 a8744e2bd55db3b8a5625c66b04b21c88f574637 This is a backport of http://review.gluster.org/8891/ Change-Id: I4504f3050674dde217e79af28cb4d2b5370fe2d5 BUG: 1148093 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8899 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
This is a backport of http://review.gluster.org/8891/

Change-Id: I4504f3050674dde217e79af28cb4d2b5370fe2d5
BUG: 1148093
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/8899
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd/quota: Heal pgfid xattr on existing data when the quota is enable 2014-09-30T16:42:40+00:00 vmallika vmallika@redhat.com 2014-09-29T07:32:30+00:00 3dcb313572082edb07ed61145570b73a898c51e2 This is a backport of http://review.gluster.org/#/c/8878/ The pgfid extended attributes are used to construct the ancestry path (from the file to the volume root) for nameless lookups on files. As NFS relies on nameless lookups heavily, quota enforcement through NFS would be inconsistent if quota were to be enabled on a volume with existing data. Solution is to heal the pgfid extended attributes as a part of lookup perfomed by quota-crawl process. In a posix lookup check for pgfid xattr and if it is missing set the xattr. BUG: 1147953 Change-Id: I707d91a056e07452bfd1e070af5eddaa752a84ac Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/8890 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
This is a backport of http://review.gluster.org/#/c/8878/

The pgfid extended attributes are used to construct the ancestry path
(from the file to the volume root) for nameless lookups on files.
As NFS relies on nameless lookups heavily, quota enforcement through NFS
would be inconsistent if quota were to be enabled on a volume with
existing data.

Solution is to heal the pgfid extended attributes as a part of lookup
perfomed by quota-crawl process. In a posix lookup check for pgfid xattr
and if it is missing set the xattr.

BUG: 1147953
Change-Id: I707d91a056e07452bfd1e070af5eddaa752a84ac
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/8890
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
snapview-server: get the handle if its absent before doing any fop 2014-09-18T18:02:31+00:00 Raghavendra Bhat raghavendra@redhat.com 2014-07-17T06:45:54+00:00 8f4c223c5f7a7a06c3b73dbb94e85d271bd84fb5 * Now that NFS server does inode linking in readdirp, it can resolve the gfid (i.e. find the right inode from its inode table) present in the filehandle sent by the NFS client on which a fop came. So instead of sending the lookup on that entry, it directly sends the fop. But snapview-server does not get the handle for the entries in readdirp (because doing a lookup on each entry via gfapi would be costly. So it waits till a lookup is done on that inode, to get the handle and the fs instance and fill it in the inode context). So when NFS resoves the gfid and directly sends the fop, snapview-server will not be able to perform the fop as the inode contet would not contain the fs instance and the handle. So fops should check for the handle before doing gfapi calls. If the handle and fs instance are not present in the inode context they should get them by doing an explicit lookup on the entry. rebase of the patch http://review.gluster.org/#/c/8324/ Change-Id: I70c9c8edb2e7ddad79cf6ade3e041b9d02241cd1 BUG: 1143961 Reviewed-on: http://review.gluster.org/8768 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
* Now that NFS server does inode linking in readdirp, it can resolve the
  gfid (i.e. find the right inode from its inode table) present in the
  filehandle sent by the NFS client on which a fop came. So instead of
  sending the lookup on that entry, it directly sends the fop. But
  snapview-server does not get the handle for the entries in readdirp
  (because doing a lookup on each entry via gfapi would be costly. So it
   waits till a lookup is done on that inode, to get the handle and the
   fs instance and fill it in the inode context). So when NFS resoves the
   gfid and directly sends the fop, snapview-server will not be able to
   perform the fop as the inode contet would not contain the fs instance
   and the handle. So fops should check for the handle before doing gfapi
   calls. If the handle and fs instance are not present in the inode context
   they should get them by doing an explicit lookup on the entry.

rebase of the patch http://review.gluster.org/#/c/8324/

Change-Id: I70c9c8edb2e7ddad79cf6ade3e041b9d02241cd1
BUG: 1143961
Reviewed-on: http://review.gluster.org/8768
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
snapview-server: register a callback with glusterd to get 2014-09-18T17:59:30+00:00 Raghavendra Bhat raghavendra@redhat.com 2014-09-18T11:42:33+00:00 82b24d64b9dc89672e6a298648f0e3959b62b1c0 notifications * As of now snapview-server is polling (sending rpc requests to glusterd) to get the latest list of snapshots at some regular time intervals (non configurable). Instead of that register a callback with glusterd so that glusterd sends notifications to snapd whenever a snapshot is created/deleted and snapview-server can configure itself. rebase of the patch http://review.gluster.org/#/c/8150/ Change-Id: Iee2582b1a823d50c79233a41cf2106f458b40691 BUG: 1143961 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/8767 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
 notifications

* As of now snapview-server is polling (sending rpc requests to glusterd) to
  get the latest list of snapshots at some regular time intervals
  (non configurable). Instead of that register a callback with glusterd so that
  glusterd sends notifications to snapd whenever a snapshot is created/deleted
  and snapview-server can configure itself.

rebase of the patch http://review.gluster.org/#/c/8150/

Change-Id: Iee2582b1a823d50c79233a41cf2106f458b40691
BUG: 1143961
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/8767
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Propagate EIO on inode's type mismatch 2014-09-16T11:26:40+00:00 Krutika Dhananjay kdhananj@redhat.com 2014-08-21T11:57:17+00:00 dfaf76cffe37183c1b8072961b085f705c1c978f Backport of: http://review.gluster.org/8574, and http://review.gluster.org/8586 Original author of the test script: Pranith Kumar K <pkarampu@redhat.com> Change-Id: I0c32bdd8e666f8175c0a8fbf940934e6ce469931 BUG: 1136830 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/8706 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
        Backport of: http://review.gluster.org/8574, and
                     http://review.gluster.org/8586

Original author of the test script:
            Pranith Kumar K <pkarampu@redhat.com>

Change-Id: I0c32bdd8e666f8175c0a8fbf940934e6ce469931
BUG: 1136830
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/8706
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>